NYC Renters’ Toolkit All the information a renter needs about their home in New York, all in one place. Uses the vast trove of publically available data at NYC Open Data.
I was particularly happy with the last project, which combined all I’d learned at Flatiron with my passion—gleaned from my wife, who represents low-income people in the Bronx in their eviction proceedings—for keeping people safe and happy in their homes. I was emotional when I received the following tweets from Finley Bomer-Lawsome (former Business Development at Flatiron) and Avi Flombaum (Co-Founder, Dean, and Chief Product Officer there):
Let me just start by saying that it’s amazing—almost overwhelming—how much data the City of New York collects, catalogues, and publicizes. Everything from the coordinates of every piece of street furniture to the holdings of the NYC Employee Retirement System (NYCERS) are available in machine-readable format at https://data.cityofnewyork.us. And they are all available through Socrata’s SODA API. The data are not organized perfectly—more on that later—but the sheer amount of it makes it an incredible tool if you can figure out how to harness it.
Naturally, I wanted to build another NYC-centric app, but what specifically? As my wife and I just moved from Brooklyn to Manhattan, I have been thinking a lot recently about housing, neighborhoods, and all the things that go along with being a part of a specific community in New York. So, it occurred to me that I should build an app that would help me look up all the information I needed (or was curious about) my new building and neighborhood.
The first group of data I make available in the app is sort of general information about the building you live in. It gets most of its information from the city’s Geoclient API:
Address: pretty self-explanatory
BBL: Building-block-boro code, one of NYC’s unique tax-parcel identifiers. Often necessary when interacting with city government.
Condo Number: Unique number assigned to each Condominium in the city.
Neighborhood: Official name of the neighborhood in which the address is located.
Community District: NYC Planning Community District in which the address is located.*
Police Precinct: NYPD precinct in which the address is located.*
Fire Company: FDNY fire company (smallest division) that responds to the address.*
School District: NYC Department of Education district in which the address is located.*
*The Community District, Police Precinct, Fire Company, and School District numbers all link to their appropriate webpages (uses regexp).
Two additional pieces of information come from the Department of Housing Preservation and Development’s (HPD) “Registration Contacts” dataset:
Owned By: The listed owner of building.
Services Managed By: The management company listed for the building.
This section shows the user all the 311 complaints and Housing Preservation and Development (HPD) violations on record for the building.
According to their website, 311 “provides access to non-emergency City services and information about City government programs.” It’s where most city residents file complaints about poor conditions in their homes or neighborhoods.
Complaints filed to HPD through 311 or other methods are investigated and, if a violation is found, it will be logged in their system and a notice of violation (NOV) will be sent to the entity required to repair it.
The requirements were further broken up as follows:
I serialized both my Address and Route models using ActiveModel::Serializers and made them available as JSON using Rails’ respond_to method.
Rendering Address index page, adding new Addresses
Adding a new Address
Showing a Route via our own API and Google Maps API
For showing a single Route, we have to get origin and destination Address information, plus use Google Maps’ Directions API to find actual routing information.
I wanted to actually load this information dynamically on a Route list page, as I thought it would be more fun. So, on the Route index page (or a User‘s nested Route index page), each Route has a “show route” button with class 'show_route', as well as a data-route-id tag for tracking which button is clicked.
showRoute gets the directions via Google Maps’ Directions API, and the object’s origin and destinationAddresses, and dynamically adds them to the DOM using another Handlebars template (FYI, directionsService is a Google Maps DirectionsService object):
For my Ruby on Rails portfolio project, I decided to do something using an additional API, as it had been fun to learn how to use Facebook’s OAuth one during the lessons.
As I have for my previous projects, I decided to incorporate one of my passions/hobbies: the New York City Subway system (actually, it uses all transit options in the city).
At first, I thought I would use the MTA’s set of APIs. However, I quickly realized that that would not allow me to provide transit directions (how to get from point A to point B) in the way that I wanted, as—duh!—they only provide real-time status of trains, buses, etc.
As it turns out, Google uses information gleaned from MTA’s API, which is formatted according to Google’s GTFS specification, and uses their computing power to figure out the directions. Ok, so I’m going to use Google Maps’ Directions API. I found a nice Gem that does that—all I had to do was provide my API key.
Next, I had to begin thinking about what models I would use. Obviously, I’d need a User model:
Presence of, uniqueness of, and email-address-ness of Email address
and an Address model:
Presence of Line 1
Presence of Borough, and its inclusion in [‘Bronx’, ‘Brooklyn’, ‘Manhattan’, ‘Staten Island’, ‘Queens’]
Since I am building this app only for New York City, I didn’t need a State field. And I included Borough so users could filter addresses, a requirement for the project.
As for the has_many through requirement, it made sense to create a Route join table. A Route would belong to an Origin and a Destination (both Addresses), and a User. A User has_many Routes, and Addresses through Routes; an Address has_many Routes, and many Users through Routes.
Presence of Origin and Destination
Presence of Name, and its uniqueness at the User level
The Name field will be a user-supplied label for the Route.
The controllers are very similar to what we’ve seen before. In addition to all seven RESTful routes for each of the models listed above, I built a Sessions controller to deal with login and logout logic. I wanted user to be able to login using either email/password combination OR OAuth2 through Google, so there are both “sessions#create” and “sessions#create_with_google” routes. A Sessions helper defines helper methods that interact with the session hash.
Got to use a lot of partial views, which was fun. There are lots of different ways to display or list addresses. Of course, put the forms into partials. In addition, I moved the error-message-displaying stuff, included on each form, into it’s own partial in /app/views/shared.
The “routes#show” route is where the Google Directions API does its stuff. The array returned by the google_maps_service gem looks pretty complicated at first glance—it’s an array of hashes many levels deep—but once you get the hang of it, it’s actually pretty simple. Here’s an example of where the Subway stop at which to get off is stored:
For the CLI Data Gem Project, part of Learn.co’s Full Stack Web Development online course, I was tasked with building a CLI (Command Line Interface) to data scraped from a public website. Now I, being such a big Star Wars fan, and who am subscribed to all Marvel’s Star Wars comics, decided to go with Wookieepedia’s “Canon Comics” wiki.
Just in case you are unfamiliar with comic books, as I was before Marvel started releasing their Star Wars series, the sorts of metadata associated with each issue are:
Series title (most issues are part of a series, which is usually focused on one character or set of characters)
Issue number (chronologically within the series)
There are other data, but I decided I’d focus on the above when building my models.
Speaking of models, the main purpose of this project was to work with object in Ruby. So, after I had made the decision that I was going to make a Star Wars Comics CLI, I started thinking about what my objects would look like.
Because comic books are physical objects, and the people who make them are, well, physical humans, it was pretty easy to figure out my model scheme:
First, I would have an class called a Series, representing the various comic book series you might find in the Star Wars canon. Each Series has at least one, but up to many, constituent Issues.
Then, I will implement an Artist class, which encapsulates the idea of any artist who works on an Issue. Now, even though each artist has a different job—one writes, one draws the letters, etc.—in reality, I know I’m not going to need have each type represented as a different type of object because they don’t need unique functionality when my CLI is just going to be listing them…
Actually, since this lab is all about object-oriented programming, let’s do it! So, I’m going to create subclasses of Artist: Writer, Penciller, Letterer, and Colorist. These don’t add additional functionality to Artists, but they will allow me to show that I understand inheritance.
Man, this turned out to the worst part—I should have looked at Wookiepedia’s HTML before deciding to use it! All I can say is, its editors don’t follow best practices when it comes to HTML/CSS. No class names or classes that don’t distinguish between the things I need them to. I ended up having to use a lot of selectors based on text formatting tags written directly in the HTML, including having to use the background color of links to tell whether it linked to a story (I’m not even dealing with these in my gem) or an issue.
Also, there are a LOT of comic books, so I ran into the problem that my gem was just taking too long to scrape up all that metadata! I don’t want my user to sit there for two minutes while the app downloads information about every Star Wars comic ever! I ended up writing the code so that it only scrapes when it needs to. For instance, if the user just wants to list the issues within a particular series, my gem will scrape for Series when the user asks to look at them, and scrape for Issues within that Series only when they’ve selected it. However, if a user wants to look at issues contributed to by a particular artist, they’ll just have to wait (Artists are only assigned to Issues when Issues are scraped, so in this case all Issues need to be scraped)!
All those things are going to pose a lot of questions to you about what kinds of projects you want to work on, what field you want to get into, or even how much money you want to make. And of course, those are all very important considerations to make.
I feel like one thing often gets lost, though, when thinking about real beginners: what kind of learner is he or she? How is a true beginner really supposed to know whether they want to get down into the mallocs and pointers with C or abstract everything away with Python and Ruby? Doesn’t it depend on how his or her brain works? I feel like it was good for me to work from the ground up, abstracting as we went, probably because I really like knowing the how and the why of things. Others probably don’t care about that and just want to get to the cutting edge as fast as possible.
I could be way off here, but there needs to be a better way for us to help programmers get started than just telling them what the “best languages” are. And I don’t know what the answer is here, or how to do this practically, I feel like we need to take a closer look at where a beginner is coming from, not just where he or she wants to go.
After basing my first project on my love of Star Wars, I decided to base the second, the Sinatra project, on another thing close to my heart, my cats. My wife and I have two, Bronx and Coco Villanelle. They’re not related, but they act like young siblings—one moment, they’re cozying up to one another on the bed; the next, we’ll find them roughhousing like little wrestlers.
My wife and I travel quite a bit, so we often have to find temporary homes for Bronx and Coco Villanelle, and we use a service that schedules everything through phone calls and emails—a little outdated in today’s app-dominated world. With that in mind, I decided my project would be a pet-sitting scheduler app!
Ok, let’s take a look at what kind of models our app will have. First, we have to have Users, because, well, people have to be able to login and use the app! And I’m actually going to have to kinds of Users, Owners and Sitters. Here, I’ve made them subclasses of Users, because there’s a lot of functionality they share.
Next, we have Pets. Just as in the real world, a Petbelongs_to an Owner, and an Ownerhas_manyPets. I know some people have just one pet, but they’re really missing out.
And finally, we have Appointments. Representing scheduled pet-sitting sessions, an Appointment belongs_to both a Pet and a Sitter.
Oh man, in my React/Redux final project, which uses New York City’s open data to create a dashboard for city renters, I ended up having to chain a lot of API GET requests together because of the way the data was organized.
For instance, to get a building’s Registration Info (information about building owner and manager), I had to first lookup a building’s Building Identification Number (BIN) in one dataset, cross-reference than BIN with a Registration ID in another, and use the Registration ID in a third to finally come across the Registration Info.
And all that resulted in (assume fetchBin, fetchRegId, and fetchRegInfo are the appropriate API calls:
I just don’t like it. Maybe some people like chaining these Promises because it’s like a timeline or something, but I think it just looks bad!
async allows you to declare a function that will return a Promise. More importantly, async functions can contain await expressions that pauses execution while the expression is resolved. Using async/await, I could refactor the above code as follows:
To me, that looks so much better and simpler. I understand there are some difficulties with error catching when using async/await, which I’ll have to look more into. And I admit that my Promise chaining did not look as neat as the example I have above, which isn’t that bad, because I hadn’t seen the proper way to do it, as detailed here. I had been indenting with every then, which was just getting crazy…
Anyway, at least it’s another option I now know about!
While writing my last blog post about tech-for-good and civic tech, I was reminded about how much I used to talk about the 18F and U.S. Digital Services programs of the Federal government. They were founded during the Obama administration to bring modern technologies and processes to Washington.
I wondered: what has happened to them over the last two years? Are they still thriving and innovating under a much different regime?
Wait, 18 what and the U.S. Digital who??
18F (named for the intersection—18th St and F St—in Washington, DC where they’re headquartered) is a “an office of federal employees within the General Services Administration (GSA) that collaborates with other agencies to fix technical problems, build products, and improve how government serves the public through technology.”
They partner with other agencies in the federal government to build digital services utilizing lean startup practices, they are dogmatic about developing with transparency (open-source), and they aim to leave their partner agencies with a sense of how software development looks outside the massive bureaucracy of the federal government.
Some of their earliest work includes:
https://analytics.usa.gov/, which displays Google Analytics data for many government websites. In case you’re interested, the government website with the most users at the moment I’m writing this is the USPS’s tracking page. By far.
https://myra.gov/, a website for the Treasury Department’s myRA program, which was a government-sponsored Roth IRA account.
U.S. Digital Service
The U.S. Digital Service is actually within the Executive Office of the President—the “White House”—so it has a more top-down approach. It recruits top technologists for short-term stints in the government to work on the Administration’s top priorities. It was created after th HealthCare.gov rollout disaster to prevent something like it from ever happening again.
Some of their earliest work:
https://my.uscis.gov/ (in partnership with 18F), a website for helping users navigate through the immigration process.
https://www.va.gov/, a website for Veterans to discover, apply for, track, and manage their benefits.
IRS Secure Access, which allows taxpayers to access their tax transcripts online, rather than through the mail.
Changes under Trump
With Trump coming into office, many were worried that these programs’ futures were grim at best. And while there certainly have been changes and large reductions in staff—at least at 18F—both still exist and are doing good work! The U.S. Digital Service is actually still run by an Obama-era hire!
According to Fast Company, 18F’s staff headcount has shrunk by at least half since Trump’s inauguration, and many staff resigned in protest of Trump’s policies and personal behavior. However, there are still civil servants there hard at work, making sure the good work started under Obama continues. In fact, just this month, they released the second version of their design guidelines for new government websites—the United States Web Design System 2.0.
U.S. Digital Service
Because the U.S. Digital Service is focused on advancing the priorities of the White House, I am more surprised to find that it is still operational, and even more surprised that it’s being run by Matt Cutts, who was there during the Obama administration. And it looks like I’m not alone in that thinking, with numerous articles referencing it (just search “matt cutts trump”). It appears the common response is that their crew is actually nonpartisan, working for the “American People.”
And they have done some great things, like saving the VA $100 million by streamlining their cloud infrastructure processes, and helping to build anti-drone defenses. And right now, they’re working on upgrading Medicare’s payment system, which is apparently 40 years old and written in COBOL and assembly.
But it’s weird to see that they’re working on helping immigrants and refugees, when that’s clearly anathema to the White House. I hope they can continue to do that kind of work without Mr. Trump ever finding out about it!
What is the self? Philosophers have been pondering that question for centuries. Luckily, we don’t have to worry about such complex contemplation when we consider self in object-oriented programming (also called this or Me, depending on the language).
Recap: object-oriented programming
Object-oriented programming is a paradigm in which state (data) and behaviors that operate on or with that data are encapsulated in objects. In a way, objects in programming are similar to real-life objects—they have attributes and can do different things depending on those attributes.
A class is a template for creating objects, and contains definitions of the data formats and available behaviours for all instances of itself. For instance, if we were representing a car in code, we could have a Carclass that defines all car objects as having attributes, such as color, horsepower, weight, etc., as well as behaviors, like accelerate, that would act differently depending that object‘s specific attributes:
attr_accessor :color, :horsepower, :weight
attr_accessor allows us to get and set the attributes of an object. Here’s it’s letting us get or set a Car‘s color, horsepower, and weight.
When we’re defining the behaviors in a class, we need to have a way to access the data of a specific instance of that class (an object). In Ruby, as in many other languages, we use the keyword self. Objects are self-aware!
In our Car example, let’s say we want to define the accelerate behavior (method in Ruby) as returning the specific Car object’s horsepower divided by its weight (a vast simplification). We would define it like this:
#sentence? Returns true if the String object it’s called on is a sentence (ends with a period).
#question? Returns true if the String object it’s called on is a question (ends with a question mark).
#exclamation? Returns true if the String object it’s called on is an exclamation (ends with an exclamation point).
#count_sentences Returns the number of sentences in a String object. In this case, a “sentence” can end with a period, question mark, or exclamation point.
Remember, we can use the self keyword in our method definitions to refer to the String object on which it is called. Let’s look at an implementation of our first three methods (hint – use the #ends_with? method to check the last character of a String):
For our final method, we just have to count the number of periods, question marks, and exclamation points, right?!?!?!? Well, that would work as long as people don’t use multiple marks in a row, like I just did. Instead, let’s count the number of substrings that end with one or more of these marks. For that, we can use our friend the #split method and a regular expression.
Do you remember how you’d write a regular expression for “one or more of periods, question marks, or exclamation points?” I recommend Rubular for checking RegExes in Ruby. Don’t continue until you’ve figured it out!
Yep, it’s /[.!?]+/. Note that you don’t need to escape characters inside the brackets. So, our final method inside the String class would look like this: