Jessica Lord, open web developer—designer

Node & npm Basics via a Simple Module

  • Jessica Lord
  • 2014-09-06
  • Node.js, npm, Development

The other day I wrote a simple Node module and thought it may be useful and digestible enough to look into and learn about some Node and npm Basics. If you have some questions about this module, you can open an issue on the repo.

Note: The module is github.com/jlord/cli-boilerplate and since this posting it's been updated (more features!) so I created a branch called tutorial on the repository, here, that matches the state of the project at the time of this writing to refer to as you read.


Before anything, of course you'll need Node.js and npm (which comes with the Node install). Now, begin!

Start with a Problem

When I start a new website I usually go and dig up some boilerplate HTML. I wanted to make "dig up" a little less work and since the boilerplate isn't but a couple dozen lines or so I thought it would be nice if I could just type a command and have it copied to my clipboard so that I could paste it into a new file. This is how cli-boilerplate was born.

In terminal there is already a command for sending text to the clipboard and with Node we can read contents of files and execute terminal commands. So, let's begin!

Initialize

npm (nice people matter) is Node.js's package manager. It's what you use if you want publish your project so that others can easily use it. It's a registry of all the published modules and their versions. You can see the latest activity here npmjs.org (at time of writing, 93,291 packages!). You can write projects in Node and run it on your computer without npm, but if you intend to publish it as we do, you'll use npm.

npm init

Create a new folder for your project to live in. Now in your terminal, navigate to that folder and run npm init. This kicks off a process that gets your package.json file started for you.

npminit

This file is like the title page, glossary, index and copyright page of a book rolled into one. You can get a sense of these files, and the JSON format, by checking out the package.json in existing projects (see here, here and here).

# make a new folder
$ mkdir cli-boilerplate
# go inside of folder
$ cd cli-boilerplate
# initialize!
$ npm init

You can fill out the information in terminal or hit enter and fill it out later in your text editor. When it's done you'll have a package.json file in your folder.

Beyond the basics

Because I want to create a module that runs from the command line, I need to add a section in my package.json called bin which will allow me to define what file gets run when x word (command) is typed into terminal. I chose boilme. Haaa. I also added where the repository and issues are on GitHub as well as keywords for when people search npmjs.org.

{
  "name": "cli-boilerplate",
  "version": "1.0.1",
  "description": "add html boilerplate to you clipboard for pasting wins",
  "keywords": ["HTML5", "boilerplate", "cli", "command line"],
  "repository": {
    "type": "git",
    "url": "http://github.com/jlord/cli-boilerplate.git"
  },
  "bugs": {
    "url": "http://github.com/jlord/cli-boilerplate/issues"
  },
  "main": "index.js",
  "bin": {
    "boilme": "./index.js"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Jessica Lord",
  "license": "BSD"
}

Create the boilerplate

We want to create the boilerplate HTML that we'll have copied to the clipboard for us when this module is used. This is easy as creating a new HTML file and filling it with the boilerplate:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="description" content="">
    <meta name="author" content="">
    <title>Title</title>
    <link rel="icon" type="image/png" href="favicon.png">
    <link rel="stylesheet" href="assets/style.css">
  </head>
  <body>
  </body>
</html>

Write the code

Now we'll create the actual code in a file named index.js, which is a common name for your main file. Below is the full code, it's pretty short! A lot of node modules are simple and do just one thing, like this one. We'll go through each part.

#!/usr/bin/env node

var fs   = require('fs')
var path = require('path')
var exec = require('child_process').exec

fs.readFile(path.join(__dirname, 'boilerplate.html'), function read(err, data) {
  if (err) return console.log(err)
  var command = "echo '" + data.toString() + "' | pbcopy"
  exec(command, function clipboarded(err, stdout, stdrr) {
    if (err) return console.log(err)
    console.log("Copied!")
  })
})

The Environment

At the very top we use #!/usr/bin/env node. Remember in package.json we set "./index.js" as the file we wanted to run after someone types boilme? Using this line at the top of index.js tells the computer when it starts to read it that it should execute — #!/ (shebang) — the code below it and do so using Node.

Require other modules

Node.js comes with a core set of modules that help you do lots of things. If there is something that is outside of the scope of core, that's when you'll use npm to find something created by someone else.

This project, being simple, just uses core modules. In this section we are requiring the core modules we want and giving them a name:

var fs   = require('fs')
var path = require('path')
var exec = require('child_process').exec

When you require a module, all of the functions inside of it become accessible to you in your code through the name you set for it. For instance, I can use the things that come with fs when I use the variable fs in my script.

fs is a module for the file system, letting us read or write files (and more!) on a computer

path is a utility for working with file paths (locations)

child_process allows us to run another process, outside of the one we're already in. Using the exec part of it allows us to specifically run a shell (command line) process

Action!

Next we'll actually use this modules to:

  1. Read the contents of the boilerplate HTML file.
  2. Create a string of the command we'd type in terminal if we were manually giving it text to send to the clipboard.
  3. Use exec to create a shell process and give it our command to run.

Let's break it down some more. Here is all the code:

fs.readFile(path.join(__dirname, 'boilerplate.html'), function read(err, data) {
  if (err) return console.log(err)
  var command = "echo '" + data.toString() + "' | pbcopy"
  exec(command, function clipboarded(err, stdout, stdrr) {
    if (err) return console.log(err)
    console.log("Copied!")
  })
})

Read file

We use fs to read the file, it wants to know where the file is and what to do after it has read it, the callback. The callback is a function that is run after the main task (reading the file contents) has completed. It will automatically get passed in first an error, if there was one, and the the contents of the file, but buffered, which means numbers (we'll change that later). Here's what the basic format looks like: fs.readFile('filelocation', callback).

The boilerplate.html will be located wherever this module exists on the users computer, so we'll use __dirname to get the location of the module on the computer and path.join to add boilerplate.html to that. That gives us the full address of the file on the computer.

Then we pass in our callback, a function we create called 'read' because it's going to get run when the file has finished being read. Next is a common pattern in Node -- we check to see if there were any errors reading the file. If there were, we want give up because there's no use doing the rest of the work. So if there is an error, we return, but we print with console.log()out what the error was.

But, if there were no errors, we'll move right along!

Run copy command

We create a string to contain what the full command would be in terminal if we were doing this manually. In Mac terminals you can type echo "Hello World!" and it will print out the string you gave it. You can feed that text directly into another function using a "pipe" which is this "|" character. We want to feed it into the copy command which is pbcopy on Macs. So typing echo "Hello World!" | pbcopy will put "Hello World" in our clipboard and and the next time we paste, we'll get "Hello World".

Since fs.readFile returns a buffer of numbers, we'll use the JavaScript method .toString() to turn it into the letters and spaces we recognize.

Using exec we spawn a new independent shell process to which we can give our shell command. It also takes in a callback, a function for what to do after it's run that command. We want it to quit and print the error if there was an error, but if there was no error, print "Copied!".

A Possibly Non-Helpful Diagram

diagram

Try it out and Publish

Now that the file is done, we can test it by running it on our computer. If we're in the cli-clipboard directory we can type npm link so that npm will use this version we've made as the global module.

$ cd cli-clipboard
$ npm link
# it will print text about the linkage
$ boilme
# it should work!

Next publish what you've done to your npm account (if you don't have one, set one up):

$ npm publish

Tada

That's the whole thing! We have a package.json so that we can register our module with npm and others can use it, we have our boilerplate.html file with our dream boilerplate, and we have the itty bitty index.js script to make what we want to happen, happen.

This post is way longer than I thought it would be. I tried to save space and didn't mention it, but you should also being using Git when you're building something and checking-in often to save your work and push it to GitHub.

Other files

The project also has a README.md and LICENSE.md file. They're not required to make a module, but they're really important files to have in a project. The former tells people how to use your project and the later shows the license the project is under. Some places can't use un-licensed projects, so it's important to add one!

365 days a GitHubber

  • Jessica Lord
  • 2014-05-15
  • GitHub, Development

I've been at GitHub a year! That's 365 days of learning, laughing and working on something I care about. That's lucky!

A little over two years ago I was an urban designer for a lovely city on the east coast. Then I spent 2012 as a Code for America fellow and fell hard for open source and GitHub.

Open source and GitHub were the community and mindset missing from my way of working before, only I didn't know it then. I did know, however, that I wanted to build more and ship more (only I didn't know it was called 'ship' then). Hello, GitHub!

Now not only do I get to use GitHub everyday but I get to be a part of making GitHub as great as it can be — the tool, the company, the community. And I get to do this with an amazing group of people that are talented, passionate and hilarious. They inspire me with code and wit every day.

I work in particular on making GitHub.com better for new users and that gets me pretty jazzed. I remember quite well what it is like to be a new user and new to open source. I remember searching (and still do — because, never stop learning) for resources online to learn new things. I want to help others improve the way they work with GitHub, I want others to come to GitHub and start contributing to open source. I love working on the tools to help people do that.

The more people out there building things and working together, the better.

But the big, big thing for me working at GitHub has been that, more than I realized could be possible at the place you work, I know my coworkers want to see me happy and succeeding. I am myself everyday. I truly feel appreciated and that I have impact and potential. I try not to take these things for granted.

I thought I'd return to urban design after CfA but when I decided to work at GitHub I was also deciding to keep doing this software development thing. It's crazy to think about what I know now compared to what I knew a year ago. I've learned so much. I wouldn't have been able to do so if it weren't for a company that shares my goals, champions people and gives me the freedom to shape my experience. I want to keep learning (more Node and even some Rails), keep spreading the knowledge (Patchwork nights! Guides!) and I'm proud to do it as a GitHubber.

Sheetsee.js v3.0

  • By Jessica Lord
  • 2014-02-28
  • Open Source, Sheetsee.js

penny

I've just updated Sheetsee.js with new features and all new docs. I've been waiting to ship this, it feels like, for far too long. I’m hoping now Sheetsee is at a solid enough place that I can focus spare time on making more fork-n-go examples.

I'll get straight to the most exciting point: the new sample data for the demos is of my flattened penny collection. Below are the other things.

New Features

Yay!

Tables: Pagination, Multi Table Sorting and Filtering

Tables can now be paginated so that you don’t have to display your entire spreadsheet of data in one go. Unless you want to.

Tables now supports having multiple tables with sorting and filtering. So long as you keep your table IDs straight, you’re golden.

See: table demo

Maps: Polygons and Lines

Maps now support polygons and lines. So long as you have the correct coordinate structure in your cells, Sheetsee will add them to the geoJSON it creates for your maps. More details for coordinates of lines and polygons in geoJSON are here, but briefly:

A linestring:

[-122.41722106933594, 37.7663045891584], [-122.40477561950684, 37.77695634643178]

A polygon:

[-122.41790771484375, 37.740381166384914], [-122.41790771484375, 37.74520008134973], [-122.40966796874999, 37.74520008134973],[-122.40966796874999, 37.740381166384914], [-122.41790771484375, 37.740381166384914]

A Multipolygon:

[[-122.431640625, 37.79106586542567], [-122.431640625, 37.797441398913286], [-122.42666244506835, 37.797441398913286],[-122.42666244506835, 37.79106586542567], [-122.431640625, 37.79106586542567]],
[[-122.43352890014648, 37.78197638783258], [-122.43352890014648, 37.789031004883654], [-122.42443084716797, 37.789031004883654], [-122.42443084716797, 37.78197638783258], [-122.43352890014648, 37.78197638783258]]

See: map demo.

Documentation

I redid the documentation and website for Sheetsee. Instead of one gigantic readme file/website, all of the different elements are given their own file/page. Now you can find docs, demos, tips and such all filed away nicely here in the repository and on the website.

Modules

The other big thing that has happened to Sheetsee is that I've taken each component (core, maps, tables and charts) and made them into their own Node.js module. This makes for smaller, more manageable code files and it makes it so that users can build their own version of Sheetsee with just the elements they want to use. For instance, if you're only making maps, just build a Sheetsee with the map component.

All builds come with the sheetsee-core module which contains all of the data organizing functions.

The module sheetsee is a command line module that builds your custom sheetsee for you.

If you're not interested in Node modules or building anything, that's ok, too. I'll always keep a full build (with all the components) here in the sheetsee.js repository (which from here on out exists to be the consolidated documentation and source of a full build).

If you need to file an issue for a bug or feature, try and file it on the repository with the component it relates to (they each have their own repository now):

Sheetsee.js Tips

  • Jessica Lord
  • 2013-09-15
  • web dev, open source, sheetsee.js

google maps

There are lots of ways you can take data from you Google Spreadsheet and use it in a website in cool ways. I’m going to do a few posts on such things. Neat!

If your spreadsheet contains address information, using templates (Sheetsee.js uses a form of Mustache), you can embed those elements into a Google Maps URL query string (this is the same URL that is generated when you type in an address into the search bar yourself) and create links/buttons that will open locations up directly in Google Maps on desktops and phones. Similarly you can put them in Yelp queries, too.

The basic elements are: a spreadsheet with address info + HTML template to create the query string.

Par Exemple

The Sheetsee Hack-Spots is an does such a thing. Here is the spreadsheet, with address information:

spreadsheet

In the HTML template for the table on the Hack-Spots page, the button's links look like this:

<a class="button" href="https://maps.google.com/maps?q={{address}},{{city}},{{state}}" target="_blank">View in Google Maps</a>
<a class="button" href="http://www.yelp.com/search?find_desc={{name}}&find_loc={{city}},{{state}}" target="_blank">Find on Yelp</a>

Here is the exact line of code on GitHub.

We're inserting the address, city, and state details from the spreadsheet into the structure of a query string for Google maps and Yelp. You can figure out the query string of a service by just using it (type in an address in Google Maps) and looking at the resulting URL.

With a some CSS and such, the resulting website has a table with the hack spots and a button for viewing in Google Maps or Yelp:

table

When the page builds, it creates the correct link for each row. When someone clicks on the buttons it takes them directly to the Google Map search result for that address. BAM!

Fork-n-go

  • Jessica Lord
  • 2013-08-11
  • github, pages, open source, sheetsee.js

fork-n-go

I began Sheetsee last year as an effort to make wider use of existing, free and easy to use web technologies. I'm always hacking on it and always trying to make it easier to use.

ss diagram

I think that using a Google Spreadsheet as a database can quickly and easily solve a lot of the initial barriers to getting-going that prevent even savy developers from starting even a simple website if it requires a database. More so I think that using Google Spreadsheet as a database can bring simple and easy control to many people.

Because you can easily pick contributors, viewers; share raw or connect to a Google Form - collaborating on information can become really, really easy. When connected to a templated website, the information becomes more meaningful.

Currently there is sheetsee-cache, a node.js caching version, Sheetsee, the basic client-side version which is much more accessible. Recently I've started getting excited about an iteration even more approachable, making use of the free and easy hosting through GitHub Pages.

All it takes to get a website going for a repository on GitHub is a branch named gh-pages containing web files. You also don’t need a master branch, you can have a repo with just one branch named gh-pages. Here is what I think is really cool, if you fork a project with just a gh-pages branch, you’re only a commit away from having a live version yourself. If this repo being forked is using sheetsee.js then everyone is a fork, commit and spreadsheet away from having a live website connected to an easy (a familiar spreadsheet UI and no 'publish' flow because Google autosaves) to use database that they manage (control permissions, review revision history).

Hack Spots

hack-spots

For example, I made this website to collect hack spots all over the world from friends and friends of friends (the spreadsheet is wide open, so you can add some, too!). It's using sheetsee to power the table, map and other elements of the page. Its source is in this repo, with just a gh-pages branch. To create an instance of this site for yourself all you need to do:

Now you have the same site connected to a spreadsheet that you manage - it's live and can be found at yourGitHubName.github.io/theReposName.

fork and commit

Potential

I think there could be many potential uses for web developers and designers to use this for small projects that catch their fancy, but I'm really interested in this as a way to make creating a live website with collaborative data available to people who think setting up a website is far beyond their grasp (or not free). All it takes is a designer/developer to create one working version for a use case to enable many to easily replicate it. Could this help civic organizers? Cities? Neighborhood groups?

tool lending

So I'm wondering, what are the possible use cases? For instance, I'm currently working on a site for managing lending tools in a group of friends or neighborhood. This information can be easily stored in a spreadsheet but made much more useable through a website. Any group could use the steps mentioned above and get this going for themselves in minutes. No installation and after one person forks the site, all other collaborators only need interact with the spreadsheet.

Edit: Also! There are lots of automation into Google spreadsheets possibilities through ifttt.com (which I use for the Pocket reads and Instagrams on the front of my site).

Hit me up on Twitter, email or on GitHub if you've got thoughts or ideas for this!