Jessica Lord, open web developer—designer

Fork-n-go

  • Jessica Lord
  • 2013-08-11
  • github, pages, open source, sheetsee.js

fork-n-go

I began Sheetsee last year as an effort to make wider use of existing, free and easy to use web technologies. I'm always hacking on it and always trying to make it easier to use.

ss diagram

I think that using a Google Spreadsheet as a database can quickly and easily solve a lot of the initial barriers to getting-going that prevent even savy developers from starting even a simple website if it requires a database. More so I think that using Google Spreadsheet as a database can bring simple and easy control to many people.

Because you can easily pick contributors, viewers; share raw or connect to a Google Form - collaborating on information can become really, really easy. When connected to a templated website, the information becomes more meaningful.

Currently there is sheetsee-cache, a node.js caching version, Sheetsee, the basic client-side version which is much more accessible. Recently I've started getting excited about an iteration even more approachable, making use of the free and easy hosting through GitHub Pages.

All it takes to get a website going for a repository on GitHub is a branch named gh-pages containing web files. You also don’t need a master branch, you can have a repo with just one branch named gh-pages. Here is what I think is really cool, if you fork a project with just a gh-pages branch, you’re only a commit away from having a live version yourself. If this repo being forked is using sheetsee.js then everyone is a fork, commit and spreadsheet away from having a live website connected to an easy (a familiar spreadsheet UI and no 'publish' flow because Google autosaves) to use database that they manage (control permissions, review revision history).

Hack Spots

hack-spots

For example, I made this website to collect hack spots all over the world from friends and friends of friends (the spreadsheet is wide open, so you can add some, too!). It's using sheetsee to power the table, map and other elements of the page. Its source is in this repo, with just a gh-pages branch. To create an instance of this site for yourself all you need to do:

  • Create a Google spreadsheet with the same headers (just copy and paste header row from the original).
  • Fork the original source.
  • Edit the HTML file directly on GitHub.com to replace the original spreadsheet’s unique key with your spreadsheet’s key (found in your spreadsheet’s URL).
  • Commit your change.

Now you have the same site connected to a spreadsheet that you manage - it's live and can be found at yourGitHubName.github.io/theReposName.

fork and commit

Potential

I think there could be many potential uses for web developers and designers to use this for small projects that catch their fancy, but I'm really interested in this as a way to make creating a live website with collaborative data available to people who think setting up a website is far beyond their grasp (or not free). All it takes is a designer/developer to create one working version for a use case to enable many to easily replicate it. Could this help civic organizers? Cities? Neighborhood groups?

tool lending

So I'm wondering, what are the possible use cases? For instance, I'm currently working on a site for managing lending tools in a group of friends or neighborhood. This information can be easily stored in a spreadsheet but made much more useable through a website. Any group could use the steps mentioned above and get this going for themselves in minutes. No installation and after one person forks the site, all other collaborators only need interact with the spreadsheet.

Edit: Also! There are lots of automation into Google spreadsheets possibilities through ifttt.com (which I use for the Pocket reads and Instagrams on the front of my site).

Hit me up on Twitter, email or on GitHub if you've got thoughts or ideas for this!

Dat

  • By Jessica Lord
  • 2013-07-03
  • web dev, open data, science, gov

dat diagram

Disclaimer: I know @maxogden very well, but that doesn’t mean I like everything that he does. But I do like Dat.

It's really exciting to see all the discussion that's started based on one GitHub readme. The more perspectives, the better Dat will be. In the back and forth I've noticed a few misunderstandings about Dat and as it often goes for me, I felt the urge to diagram.

The diagram above illustrates that Dat will be an ecosystem of modules that can be tacked on and synced with different datasets; importantly, this includes very big datasets. Modules and combination of modules can be shared. These modules will use a common API but can be written in any language and hosted anywhere. The ecosystem exists for the benefit of data consumers and lives outside the realm of data producers.

Dat and Government

Dat is being built to solve a problem that exists today without a good solution: collaborating on really large datasets and sharing data transformations. While Dat can be used by anyone and government certainly fits the anyone bucket, it isn't a new tool for government. It is a new tool for those who work everyday with data, government or other.

Because Dat will be a platform that works with any type of tabular data, it prefers no one type, it is completely independent of the government data standards and liberation efforts that are ongoing. And yes, this effort is a cultural effort. I know this to be very true as I've been involved with government all of my career from the City of Boston to Code for America (apply now!) and am still active in this important cultural shift.

Dat will exist outside of and in parallel to whatever comes of this effort and therefore will work no matter what standards emerge from the effort; that's awesome!

Build Dat

Because nothing like Dat exists today, I absolutely believe someone should give something like Dat a go. I believe we should always try to make things -- because things make other, better things possible. E.O Wilison says it much more eloquently:

The reason is that the adoption of any one innovation made adoption of certain others possible, which then, if useful, were more likely to spread. Bands and communities of bands with better combinations of cultural innovations became more productive and better equipped for competition and war. Wilson, Edward O. (2012-04-02). The Social Conquest of Earth (Kindle Locations 1496-1498). Norton. Kindle Edition

Git

Remember, Git wasn't the first version control and most of us wouldn't know it like we do today if weren't that years after it was built someone built an interface on top of it.

While Git works for many of us, it doesn't work for everyone. And those that it doesn't work for are doing awesome work in the civic, science or legal space. They have huge, huge datasets that move slow on Git. They have realtime data constantly updating and yielding too many commits for Git to handle. They have no good way to share this data or the transformation algorithms they used. This transformation algorithm aspect is really awesome because it also helps the majority of us who work with small or medium sized datasets that are routinely published in the same file and text formats that require frequent transforming. With Dat you can create a transformation module in your favorite language that can be hooked up to your data sync so that you don't have to constantly re-transform -- and you can share that modules with others. It's simply a script hosted somewhere. There is a place for hosting code that I love.

Open + Community Built

I'm excited that Dat will not only be built in the open but embody its own ideals and be structured so that contributing is easy. Similar to NPM, the package manager for the Node.js community, Dat will be made up of small modules related through a common API. The community can submit modules to transform data or submit pull requests to the API. There are already pull requests and issues (and forks?!) on the readme.

<3

Dat doesn't change how groups are publishing data -- it changes how people interact with it and share it with others. I can't wait to see what comes from all the brains on this!

Sheetsee.js v2.0

  • By Jessica Lord
  • 2013-04-03
  • web dev, sheetsee.js, spreadsheets, js

sheetsee 2

ANNOUNCEMENT!

Thanks to a Code Sprint Grant from Knight-Mozilla's Open News I'm going to spend lots of time making sheetsee.js all that it can be!

Sheetsee.js is Google spreadsheet and visualization mashup I built as a part of my project, See Penny Work, during my fellowship with Code for America last year. The goal for sheetsee.js is to use Google spreadsheets as an easy way to store, serve and collaborate on data all the while feeding it into a website that visualizes the data. Once the website and spreadsheet are connected (set it and forget it you only need to interact with the spreadsheet; add another row and with every autosave your website reflects the changes. No pushing or uploading, no teaching collaborators anything more than the spreadsheet.

Currently it's built really specifically for its original instance in my CfA project. I'll be spending the next two months making it much more general and adding features like more visualizations out of the box, data caching, data filters, responsiveness and other exciting stuff.

It will be built with news organizations in mind -- a template for small and medium newsrooms to customize and be able to deploy quickly. And WBEZ is the guinea pig! But, as I mentioned, I'm making it really general and can I imagine many other instances in communities or small groups where this would be useful (in particular, community collected data). I've got some side projects in mind for it, too. Related, I'll be at coffee shops in Oakland a lot in the near future, join and hack!

Lots of thanks to Dan Sinker for making things happen!

Check out the awesome stuff Open News does!

A Government More Like the People

  • By Jessica Lord
  • 2013-04-01
  • open gov

gov gulf diagram

A year out of college and doe-eyed I started work in the public sector. Jaded by the starchitecture, design-for-designers mentality of my architecture schooling, I wanted to work at city scale and for the people. It wasn’t long, however, until I realized I wasn’t like most people in city government. A young go-getter, I was teaching coworkers the tools I’d been using throughout and before college; it was an office joke when, after a year and a half I asked for a performance review (a coworker said he hadn’t had one in the 15 years he’d been there). I began to work across departments as I was the only one who could do some of the things I could, and there was no one else I could ask, “Is this the best way to do this? Have you tried this new tool?”

After three years I was worried about the amount of non-learning that had happened in my time there, I was exhausted from trying to make change only to hit wall after wall. Luckily, this coincided with my meeting some 2011 Code for America fellows. Less than a year later I’d be in California for the first time and working on government technology from a different, and more personally satisfying, angle.

There is a great gulf between where we are as a society in terms of technology, connectedness and new efficiencies and where government sits in respect to the same. This gulf has become apparent enough that it is the subject of books, talks, fellowships – it’s now a movement.

While after almost 4+ years willfully in the trenches of government technology I have more war stories than success stories, I’m still optimistic. This is because we’re at a unique place. A revolutionary kind of place. For the first time we the people have the capacity to hold government accountable in a meaningful way. The tools we use in our everyday lives often empower us to understand our cites better than those at City Hall.

The movement is a result from the pressure the gulf places on government. Government hasn’t put pressure on itself to get better – this movement comes from the people. I believe we’re at the beginning of a sea change.

There are many reasons government hasn’t kept up: tight budgets, the expectation to be unflappable and the correlated risk aversion, siloed departments, aging staff and skills and the lack of competition. These reasons and more are being directly challenged by the fact that our lives outside of government have been dramatically changed by the technology of the last 20 years.

Tight Budgets : Open Source, SaaS, Google et al

The open source community is thriving – there’s even an civic tech open source community. The software being built is good and if it’s not free it’s sold as a service for what’s still a bargain. It’s often continuously maintained – that means free upgrades. Why on Earth would a cash strapped city still shell out hundreds of thousands of dollars for long term binding contracts on software that’s probably out of date as soon as they get it?

Unfortunately, even if you get a city that wants to go the open source route, procurement standards hold them hostage and keep small companies out of competition with enterprises. People are working on that.

An even brighter area of possibility is all of the free services many of us take for granted that haven’t yet penetrated City Hall. I’m talking about making maps, charts and forms through Google, creating a website for a project or department on WordPress or Tumblr. Storing and cataloguing photographs on Flickr. These services are tried and true at this point, are free or almost free and best of all make it really, really easy to share.

Risk Aversion : Prototypes

The government must remain to some degree risk adverse – private information must be secure and City Halls mustn’t crumble. But there is enough room to play in to keep everyone busy for a while.

If New York City’s Broadway pedestrian zone hasn’t already made this clear: some street paint can go a long way. Low cost, temporary changes are a great way of making positive change or staying away from costly, permanent mistakes. And no doubt the quicker action makes for a happy citizen, too. A more agile government can benefit all.

Sites like Neighborland, Neighbor.ly and Better Block are full of many relatively (compared to typical city projects) easy to initiate ideas and some of the free services mentioned earlier can help facilitate prototype projects.

Competition : Be the Best!

Cities do not have to compete to be the government for the city. And if you’re not the Mayor there is a good chance you can keep your job for 20, 30 years whether you learn new things or not. So while private firms handling tasks similar to those of cities like urban planning, payments and record keeping kept on top of changes in standards, City Hall did not. I think this is a large part of why we are where we are today.

Be a more innovative, open and agile city than that other city. Sick as I am of these buzzwords already, I think the movement has finally given cities something to compete for, lest be publicly shamed and left behind. Hopefully in time the buzz-factor will yield real institutional change by promoting innovation and professional development.

I hope that this happens and eventually the insides of City Halls become more attractive to developers and civic tech becomes exciting for entrepreneurs.

Silos : Distributed Source Control

It’s sad and it’s frustrating how many situations boil down to lack of communication or turf wars between departments (the former probably leading to the latter). If only staff and departments could contribute to databases, contracts and other documents the way programmers use GitHub. And! What if they were by default open source. At minimum, shared access across all departments for employees.

Connectedness : @Gov

The best thing about the internet is the ease in which it allows us to share. Your friends on Twitter and Facebook may share too much, but nonetheless, social networks have made people and companies more accessible and accountable (if only for fear of Twitter storms). We want the same from our government – clear and simple ways to reach who we need.

I don’t expect or think that government should ever be on the bleeding edge of technology – that is not their role. But the current gulf is too great. I want a government a little more like the people: open source and distributed like nerds, agile and always looking to improve like a startup and connected and accountable like friends and family.

Horizontal Web

  • By Jessica Lord
  • 2013-01-24
  • web dev, design

diagram

I recently re-did Mikeal Rogers website, Future Aloof. As the person who came up with the designs, I have only myself to blame for the crazy mess that is the horizontal reading website I’d envisioned. Let’s talk about it. Below is the structure of the important components for Future Aloof:

key

CSS Multi-Column, Video Bar

I needed the text to expand right in columns in a predetermined area of the page subject to change depending on if the video bar is toggled on and off. Video bar should be the width of the site and not end awkwardly in the middle. The path I took to making this work is what I discuss below, ignoring a lot of the overflow, white-space and other tribulations. The site also does a few other things like respond to your screen and at 1024px and under the video bar is eliminated and ‘video’ in the menu no longer toggles the bar but becomes a link to a video page. But those weren’t as finicky as these other bits.

Issue: Force Columns to Expand Right

Multi-column, and it seems the web in general, wants to expand down – not right. The only way I found to constrain the multi-column element #entries where I wanted it and force it to expand right was to wrap it in a container #entryWrapper with padding . There is a max-height option, which I may or may not have tried, it’s a haze, but in the end I chose to keep all constraining forces together in #entryWrapper.

On #entries I’m using column-width: 300px; so that it generates as many columns as it needs (rather than prescribing a number of columns with column-count: x;). Note, it uses this as a min-width suggestion and in my case generated columns that are 387px wide. I’m happy to say that the column-gap will remain true to you.

Issue: No Self Awareness

Now that we’ve got the columns constrained to where we want them on the page and they’re expanding right, we must solve the issue that it #entries has no idea how many columns it’s generated and how wide that’s made it or its parent #entryWrapper. This means other children like #videoWrapper won’t fill to match this unknown width.

So I set empty span “markers” at the start #theStart and end #theEnd of the text in #entries. I then used jquery to find the distance between the two, adjusting for negative numbers if the page has been scrolled right and the remaining column width to the right of #theEnd.

var start = Math.abs($('#theStart').offset().left)
var end = ($('#theEnd').offset().left) + 400
var pageWidth = start + end

I use this the give #videoWrapper a width on load and when the video bar is toggled on and off (because when the text has more or less vertical space, it takes up more or less horizontal space).

Also, forget having any control over things like particular columns or every other column or nth column…

Issue: Firefox

While Firefox supports the multi-column in general, it doesn’t seem to support column-break-before or column-break-after, which is a shame, because in webkit that’s how you can tell it to avoid breaking in the middle of lists or block quotes and to always break for new posts.

Those damn scrollbars. In webkit I can hide the scrollbar ::-webkit-scrollbar {height: 0px; width: 0px;} and keep things looking the way I want while still having scrolling available through trackpad, mice, arrow keys or on-screen arrow clicks. Since the scrolling element is behind the right and left column navigation elements, in Firefox the scrollbar was awkwardly cut off on each end and made the center portion of the page however many pixels taller than the two right and left elements. Not wanting to use major plugins for dealing with scroll bars I sacrificed some of the single-frame experience of the page and used jquery to check for Firefox and then change the CSS structure so that the entire page scrolled and a more tolerable-full-screen-width scrollbar is used.

Tell Me There is a Better Way

Things seems fairly straightforward now, but it was quite a process. I did a lot of internet-researching, friend’s brains picking and general starting over and trying again before I came to these solutions. There doesn’t seem to be many horizontal reading-based (not just photography portfolios) websites out there – perhaps there is a reason. I’ll also mention that I was very much trying to keep the site as light and simple as I could. I didn’t want to bulk up on plugins or write mega js to parse text and generate columns. I like it being mostly CSS and 69 lines of jquery (half of which is unrelated to columns and scrolling right but handles toggling of the video bar and menu navigation) But, alas, there now exists one! A horizontal reading site! Push the web right!

But seriously, share experiences and insights, please.

Links

  1. Mikeal Rogers’ Future Aloof
  2. Multi-Column Layout on MDN
  3. Thinking for A Living, the only really good, not strictly photos, horizontal website I know of