Thanks to a Code Sprint Grant from Knight-Mozilla’s Open News I’m going to spend lots of time making sheetsee.js all that it can be!
Sheetsee.js is Google spreadsheet and visualization mashup I built as a part of my project, See Penny Work, during my fellowship with Code for America last year. The goal for sheetsee.js is to use Google spreadsheets as an easy way to store, serve and collaborate on data all the while feeding it into a website that visualizes the data. Once the website and spreadsheet are connected (set it an d forget it!) you only need to interact with the spreadsheet; add another row and with every autosave your website reflects the changes. No pushing or uploading, no teaching collaborators anything more than the spreadsheet.
Currently it’s built really specifically for its original instance in my CfA project. I’ll be spending the next two months making it much more general and adding features like more visualizations out of the box, data caching, data filters, responsiveness and other exciting stuff.
It will be built with news organizations in mind – a template for small and medium newsrooms to customize and be able to deploy quickly. And WBEZ is the guinea pig! But, as I mentioned, I’m making it really general and can I imagine many other instances in communities or small groups where this would be useful (in particular, community collected data). I’ve got some side projects in mind for it, too.
Related, I’ll be at coffee shops in Oakland a lot in the near future, join and hack!
Lots of thanks to Dan Sinker for making things happen!
A year out of college and doe-eyed I started work in the public sector. Jaded by the starchitecture, design-for-designers mentality of my architecture schooling, I wanted to work at city scale and for the people. It wasn’t long, however, until I realized I wasn’t like most people in city government. A young go-getter, I was teaching coworkers the tools I’d been using throughout and before college; it was an office joke when, after a year and a half I asked for a performance review (a coworker said he hadn’t had one in the 15 years he’d been there). I began to work across departments as I was the only one who could do some of the things I could, and there was no one else I could ask, “Is this the best way to do this? Have you tried this new tool?”
After three years I was worried about the amount of non-learning that had happened in my time there, I was exhausted from trying to make change only to hit wall after wall. Luckily, this coincided with my meeting some 2011 Code for America fellows. Less than a year later I’d be in California for the first time and working on government technology from a different, and more personally satisfying, angle.
There is a great gulf between where we are as a society in terms of technology, connectedness and new efficiencies and where government sits in respect to the same. This gulf has become apparent enough that it is the subject of books, talks, fellowships – it’s now a movement.
While after almost 4+ years willfully in the trenches of government technology I have more war stories than success stories, I’m still optimistic. This is because we’re at a unique place. A revolutionary kind of place. For the first time we the people have the capacity to hold government accountable in a meaningful way. The tools we use in our everyday lives often empower us to understand our cites better than those at City Hall.
The movement is a result from the pressure the gulf places on government. Government hasn’t put pressure on itself to get better – this movement comes from the people. I believe we’re at the beginning of a sea change.
There are many reasons government hasn’t kept up: tight budgets, the expectation to be unflappable and the correlated risk aversion, siloed departments, aging staff and skills and the lack of competition. These reasons and more are being directly challenged by the fact that our lives outside of government have been dramatically changed by the technology of the last 20 years.
Tight Budgets : Open Source, SaaS, Google et al
The open source community is thriving – there’s even an civic tech open source community. The software being built is good and if it’s not free it’s sold as a service for what’s still a bargain. It’s often continuously maintained – that means free upgrades. Why on Earth would a cash strapped city still shell out hundreds of thousands of dollars for long term binding contracts on software that’s probably out of date as soon as they get it?
Unfortunately, even if you get a city that wants to go the open source route, procurement standards hold them hostage and keep small companies out of competition with enterprises. People are working on that.
An even brighter area of possibility is all of the free services many of us take for granted that haven’t yet penetrated City Hall. I’m talking about making maps, charts and forms through Google, creating a website for a project or department on WordPress or Tumblr. Storing and cataloguing photographs on Flickr. These services are tried and true at this point, are free or almost free and best of all make it really, really easy to share.
Risk Aversion : Prototypes
The government must remain to some degree risk adverse – private information must be secure and City Halls mustn’t crumble. But there is enough room to play in to keep everyone busy for a while.
If New York City’s Broadway pedestrian zone hasn’t already made this clear: some street paint can go a long way. Low cost, temporary changes are a great way of making positive change or staying away from costly, permanent mistakes. And no doubt the quicker action makes for a happy citizen, too. A more agile government can benefit all.
Sites like Neighborland, Neighbor.ly and Better Block are full of many relatively (compared to typical city projects) easy to initiate ideas and some of the free services mentioned earlier can help facilitate prototype projects.
Competition : Be the Best!
Cities do not have to compete to be the government for the city. And if you’re not the Mayor there is a good chance you can keep your job for 20, 30 years whether you learn new things or not. So while private firms handling tasks similar to those of cities like urban planning, payments and record keeping kept on top of changes in standards, City Hall did not. I think this is a large part of why we are where we are today.
Be a more innovative, open and agile city than that other city. Sick as I am of these buzzwords already, I think the movement has finally given cities something to compete for, lest be publicly shamed and left behind. Hopefully in time the buzz-factor will yield real institutional change by promoting innovation and professional development.
I hope that this happens and eventually the insides of City Halls become more attractive to developers and civic tech becomes exciting for entrepreneurs.
Silos : Distributed Source Control
It’s sad and it’s frustrating how many situations boil down to lack of communication or turf wars between departments (the former probably leading to the latter). If only staff and departments could contribute to databases, contracts and other documents the way programmers use GitHub. And! What if they were by default open source. At minimum, shared access across all departments for employees.
Connectedness : @Gov
The best thing about the internet is the ease in which it allows us to share. Your friends on Twitter and Facebook may share too much, but nonetheless, social networks have made people and companies more accessible and accountable (if only for fear of Twitter storms). We want the same from our government – clear and simple ways to reach who we need.
I don’t expect or think that government should ever be on the bleeding edge of technology – that is not their role. But the current gulf is too great. I want a government a little more like the people: open source and distributed like nerds, agile and always looking to improve like a startup and connected and accountable like friends and family.
I recently re-did Mikeal Rogers website, Future Aloof(actually we’re still fixing a couple of bugs, so bear with us). As the person who came up with the designs, I have only myself to blame for the crazy mess that is the horizontal reading website I’d envisioned. Let’s talk about it. Below is the structure of the important components for Future Aloof:
CSS Multi-Column, Video Bar
I needed the text to expand right in columns in a predetermined area of the page subject to change depending on if the video bar is toggled on and off. Video bar should be the width of the site and not end awkwardly in the middle. The path I took to making this work is what I discuss below, ignoring a lot of the overflow, white-space and other tribulations. The site also does a few other things like respond to your screen and at 1024px and under the video bar is eliminated and ‘video’ in the menu no longer toggles the bar but becomes a link to a video page. But those weren’t as finicky as these other bits.
Issue: Force Columns to Expand Right
Multi-column, and it seems the web in general, wants to expand down – not right. The only way I found to constrain the multi-column element #entries where I wanted it and force it to expand right was to wrap it in a container #entryWrapper with padding . There is a max-height option, which I may or may not have tried, it’s a haze, but in the end I chose to keep all constraining forces together in #entryWrapper.
On #entries I’m using column-width: 300px; so that it generates as many columns as it needs (rather than prescribing a number of columns with column-count: x;). Note, it uses this as a min-width suggestion and in my case generated columns that are 387px wide. I’m happy to say that the column-gap will remain true to you.
Issue: No Self Awareness
Now that we’ve got the columns constrained to where we want them on the page and they’re expanding right, we must solve the issue that it #entries has no idea how many columns it’s generated and how wide that’s made it or its parent #entryWrapper. This means other children like #videoWrapper won’t fill to match this unknown width.
So I set empty span “markers” at the start #theStart and end #theEnd of the text in #entries. I then used jquery to find the distance between the two, adjusting for negative numbers if the page has been scrolled right and the remaining column width to the right of #theEnd.
var start = Math.abs($('#theStart').offset().left)
var end = ($('#theEnd').offset().left) + 400
var pageWidth = start + end
I use this the give #videoWrapper a width on load and when the video bar is toggled on and off (because when the text has more or less vertical space, it takes up more or less horizontal space).
Also, forget having any control over things like particular columns or every other column or nth column…
Issue: Firefox
While Firefox supports the multi-column in general, it doesn’t seem to support column-break-before or column-break-after, which is a shame, because in webkit that’s how you can tell it to avoid breaking in the middle of lists or block quotes and to always break for new posts.
Those damn scrollbars. In webkit I can hide the scrollbar ::-webkit-scrollbar {height: 0px; width: 0px;} and keep things looking the way I want while still having scrolling available through trackpad, mice, arrow keys or on-screen arrow clicks. Since the scrolling element is behind the right and left column navigation elements, in Firefox the scrollbar was awkwardly cut off on each end and made the center portion of the page however many pixels taller than the two right and left elements. Not wanting to use major plugins for dealing with scroll bars I sacrificed some of the single-frame experience of the page and used jquery to check for Firefox and then change the CSS structure so that the entire page scrolled and a more tolerable-full-screen-width scrollbar is used.
Also, hopefully you’re on the latest version of Firefox.
Tell Me There is a Better Way
Things seems fairly straightforward now, but it was quite a process. I did a lot of internet-researching, friend’s brains picking and general starting over and trying again before I came to these solutions. There doesn’t seem to be many horizontal reading-based (not just photography portfolios) websites out there – perhaps there is a reason. I’ll also mention that I was very much trying to keep the site as light and simple as I could. I didn’t want to bulk up on plugins or write mega js to parse text and generate columns. I like it being mostly CSS and 69 lines of jquery (half of which is unrelated to columns and scrolling right but handles toggling of the video bar and menu navigation) But, alas, there now exists one! A horizontal reading site! Push the web right!
But seriously, share experiences and insights, please.
Farewell, 2012! It’s been real. Went a lot of places during my Code for America fellowship this year, but not in middle America, which is just as well, I needed a place to drop a vertical calendar in this map.
Code for America was the best decision I’ve made and this has been the best year I’ve had. That’s it and that’s all. #woowoo
It’s my San Franniversary! A year ago today I flew to California for the first time to do this CfA thing. Of course, I now live in the warmer, flatter, node.js-ier and according to me, all around nicer Oakland, but that doesn’t play as nicely with ‘anniversary’. Also, what a year!
I finally actually sat down to tackle my multiple Heroku accounts problem. I started by following Chris’s comment and using a plugin. Something went awry, probably my fault, but I eneded up uninstalling it (which turns out to be a pain) and figuring out how to do what it does manually.
Create new keys for accounts
First, create new keys for each of the accounts you’re setting up. For me that’s a work account and personal account.
ssh-keygen -t rsa -C "your@email.com"
When it asks what to name it, give it a name that’s easy to recall, like identity.heroku.work.
Configure your system .ssh
Next, you have to edit your .ssh/config file and set up new hosts. If you use Textmate, you can type mate ~/.ssh/config in terminal. Add these lines, changing out the work/personal and IdentityFile to match your needs/keys.
Now edit the .git/config files. Inside of your root directory for the repo, type mate .git/config to open the file. It will look something like this, but under [remote "heroku"] you will change the url from heroku.com to heroku.work (or whatever you’re calling your different accounts).
In Terminal, navigate to the root directory for the repo and heroku login to the account associated with that repo. You’ll need to tell heroku what key to use with that account.
heroku keys:add ~/.ssh/identity.heroku.work.pub
Repeat for other repos you have associated with that account. Then navigate to the root of a repo with a different account, make sure to now heroku login with your other account login information. Repeat the steps for adding the keys to this account (only add the other account, ie, identity.heroku.personal).
So many keys, so much fun. If you are like me and had made a hot mess of keys, it may be useful to list your keys (in heroku and on your computer), see what’s going on and clean up. Now I have one key that github uses and two keys for my two Heroku accounts and things are running much more smoothly.
I know Instagram’s just come out with web profiles, but they’re not on my page and they’re not in circles. So quick and dirty (it took longer to write this post, see note on code) I put up my photos, here, and in circles. Some weeks ago I started using an ifttt.com (which is so, so awesome) recipe that sends my Instagram photos (dates and captions as well) to a Google Spreadsheet. I’m all about Google Spreadsheets. The same spreadsheet powers the Instagram portion on my dashboard page.
The Parts
I use tabletop.js to get my spreadsheet data into usable json. Once I’ve got that data, I tell mustache.js to take it, format each piece (I’m using just the url for the Instagram photo) into a div and then place all of those divs into the holder waiting patiently for it in my html. I use CSS to format it into circles.
The Code
Don’t even get me started on trying to format code snippets in WordPress. Ugh. So, sorry there are no indents right now.
The HTML/CSS
<head>
<!-- just showing the important parts, you'll obvs have a few more things on your page -->
<!-- load in tabletop and mustache -->
<script src="/wp-content/themes/Starkers/tabletop.js" type="text/javascript"></script>
<script src="/wp-content/themes/Starkers/ICanHaz.js" type="text/javascript"></script>
</head>
<style>
/* confine your image to a circle */
.instaImgCirc img {border-radius: 1000px;}
</style>
<body>
<!-- this dude waits patiently to be filled with stuff -->
<div id="instagram"></div>
</body>
The JS
<script id="instagram" type="text/html">
{{#rows}}
<div class="instaImgCirc"><img src="{{instasource}}" width="100%"/></div>
{{/rows}}
</script>
<script type="text/javascript">
// Your spreadsheets URL
var URL='https://docs.google.com/spreadsheet/pubkey=0Ao5u1U6KYND7dGRZbTUwd3JQZ1k3OE9KTVZqZUYwZHc&single=true&gid=0&output=html';
//
function loadSpreadsheet() {
// let's get this party started
Tabletop.init( { key: URL, callback: showData, simpleSheet: true } ) }
//
// use this if you want to use a specific number of your most recent Instagrams
function getLast(array, howMany) {
start = array.length cut = start - howMany
if (start < 20) {
return array
} else {
array = array.splice(cut) return array.reverse()
}
}
//
document.addEventListener('DOMContentLoaded', function() {
loadSpreadsheet() })
//
showData = function(data) {
var instagram = ich.instagram({
"rows": getLast(data, 12)
// use "rows": data.reverse() if you want to use all the Instagrams in your spreadsheet
})
document.getElementById('instagram').innerHTML = instagram;
}
</script>
</html>
Later Ons
I’d like have my captions appear when hovering over the image and to cue up, say, just 20 photos at a time and have new photos automatically appear without refresh. Perhaps this will happen on the 12 hour plane ride I have tomorrow.
The other week I moved two WordPress sites to Heroku and their media uploads to S3. How happy it’s made me!
For the first I used this tutorial, which sets up WordPress with PostgreSQL using this Github repo - but then I moved the other site (this one!) with this tutorial and used the new-ish on Heroku ClearDB for mySQL. I used the tantan-s3 Wordpress plugin to connect to a bucket on my S3 account. And BAM.
A couple of notes:
If you’re using the second tutorial, on step 8, I changed the database creds in my wp-config.php file (not wp-admin.php).
You won’t be able to update WordPress, add themes or plugins from within the CMS when hosting on Heroku. But it’s not hard, you’ll just add the files/folders to your set of WordPress files and re-push them to Heroku.
Multiple SSH Keys
Unfortunately, it turned out to be a hot mess for me because the first site is a project for work, so it’s on a work Heroku account, the second is this site, my site, so it’s on my personal Heroku account. Actively working on both and needing different SSH keys is a pain. If you run into this, here’s what I’m doing – it’s a bit of a pain so suggestions, please. First, add a new key. Then, when you’re trying to push and it tells you:
! Your key with fingerprint blahblahblah is not authorized to access blahblah
Remove that key and add the one your stored earlier.
ssh-add -d && ssh-add ~/.ssh/heroku_rsa
DNS and Done
Pointing your domain to the site is easy, too. Heroku has the steps here. Then you’re done.
I’m currently building up this site in spare chunks of time as I finish up my Code for America Fellowship. Concentrating first on work samples and last on blog entries. But! I’m really excited to finally get myself one place to put up code snippets and thoughts about urban environments. Stay tuned for updates in the near-ish future. The Code for America fellowship ends November 15th!
