The first Array of Things sensor went online on September 1st and since then, 3 more sensors have gone online. Array of Things was deployed with a commitment to transparency and today we’ve launched an interactive map of the installed and planned sensors to be installed on Chicago’s Open Data Portal.

You can download the raw data behind the map from the portal as well.

Array of Things will continue to expand throughout the next couple of years for a total of 500 sensors. Later this year, after the data from the sensors are validated, data will be streamed online and in the open so researchers, citizen scientists, and the curious can analyze the data.

Fall is around the corner and so is school. Now there is a lot extra on the data portal for those who are interested in school data. Chicago Public Schools has posted 10 years of maps on the City of Chicago Open Data Portal, ranging from school locations, attendance boundaries, safe passage routes, Charter school “preference” and geographic network boundaries, and Local School Council (LSC) Voting Districts.

Each one of these data sets can be interactively viewed on the data portal or downloaded to make your own custom maps. The following datasets (50 altogether) are currently available on the City’s Open Data Portal:

These data compliment dozens of education data sets already on the Open Data Portal, including the location of Chicago Early Learning programs and annual progress report cards.

Details for the data nerds

There is a special treat for school data enthusiasts who want to combine and merge all of these data. Attribute table field names and data types (including test field lengths) will remain consistent for all attendance boundary and school location shapefiles. This means, for instance, that you can easily match schools listed in the Local School Council (LSC) with data in the School Locations, reducing the time and effort of trying to match school names.

There will be some exceptions. Prior to school year 2007-08, a 4-digit “UNIT_ID” format was used. For 2007-08 and on, the format switches to a 6-digit “SCHOOL_ID”. The “UNIT_ID” field has been retained for years 2007-08 through 2010-11, which can be used as crosswalk to link earlier years to later ones.

School names will vary slightly for a number of schools due to different naming conventions employed in previous years. It is best to use the “SCHOOL_ID” fields when linking/merging historical data. Some school addresses will also change as a result of relocations or co-locations with other schools over the years.

The school locations data set will also have some exceptions. School Type (SCH_TYPE) will vary slightly over the years. Generally, schools will be categorized as either an elementary, middle, high school (all District-operated), charter, contract, or alternative.
For more information, please email datagovernance@cps.edu.

With almost eleven years of historical data and more than one currently active license for every five City residents, Business Licenses is one of the most popular datasets on the Chicago Open Data Portal and a key resource for understanding economic activity.  However, a known limitation has been that the License Code and License Description sometimes do not make it clear what the business actually does.

We have now been able to go deeper into the system the Department of Business Affairs & Consumer Protection uses to manage the tens of thousands of licenses issued or renewed every year to add Business Activities (and corresponding Business Activity IDs useful for quick filtering) to many license records.  For example, the “Limited Business License” that makes up almost 40 percent of active licenses is, admittedly, not terribly descriptive.  With the new Business Activities field, one can distinguish Hair Services, Tax Preparation, Car Washes, and the many other types of business activity that are recognizable to most people potentially shopping at these businesses.

It is important to note that some business licenses cover more than one type of Business Activity.  In these cases, all available activities (and, again, corresponding codes) are listed in the columns, separated by the “pipe” character (|).

With the addition of this more-specific information on business activities of each license, we have removed addresses from some licenses where that information might unreasonably compromise privacy, currently fewer than 0.05 percent of all license records.

As a reminder of a longstanding feature that might be overlooked, the most useful records for many purposes are the currently active licenses (as opposed to expired ones). A filtered list of just these licenses is available as both a table and a map.

Powered by Socrata

City of Chicago is continue to build an outstanding technical team and there are openings ranging from senior leadership, GIS, and database administration.

Working for the city provides a number of excellent benefits, such as:

  • Make a direct impact on the 2.8 million residents and over 46 million visitors to Chicago.
  • Modernize government processes for the 21st century, which has included paperless reform, driving the use of modern tools, and
  • Understand how Chicago functions and drive changes to improve efficiency and quality of life.
  • Excellent benefits

Take a look at the openings below and apply today.

Array of Things Node

The Array of Things has reached another milestone.  This week, the final Privacy and Governance Policies were published on the project website.  The final policies, along with detailed responses to 80 comments and the engagement report, can be found at https://arrayofthings.github.io/privacypolicy.html   This step in the AoT project invited the public to provide comments and ask questions about how the data collected by the sensor nodes will be protected and managed.  They were provided with several means to engage with the City, University of Chicago and Argonne National Laboratory.  The project partners have responded to each comment and shared all of the details of the engagement process, along with lessons learned.  This information will serve as a foundation for future public engagement as the Array of Things nodes are installed across the City.

Building Violations DataLense

 

One of the most defining characteristics of any city is its buildings. They help define the physical character and aesthetic of any city. A USA Today reader poll put Chicago’s skyline as the second-best in the United States. Buildings go beyond the downtown and into our neighborhoods to serve a greater function to also house and shelter us. So it’s also important to make sure these buildings are safe and cared for, which is accomplished by the City of Chicago’s Department of Buildings which inspects and sometimes issues violations for buildings failing to meet a standard for quality and safety.

Several years ago, Chicago published building violations data on the data portal. Last year, the city began publishing a list of problematic landlords and a list of building code scofflaws. Today, we’ve expanded building safety data on the open data portal by releasing administrative hearings data for ordinance violations issued by the Department of Buildings on the open data portal.

More…

"Taste of Chicago Blue Hour" by Justin Kern and licensed under Creative Commons Attribution-NonCommercial-NoDerivs 2.0 Generic (CC BY-NC-ND 2.0)

Chicago finished another phase on its commitment for paperless permits. You can now apply for business permits completely online with no need to follow-up at City Hall.

Chicago’s food inspection team uses a free, open source predictive algorithm to dispatch food inspectors. But it’s when a small start-up used the code for other governments is when it got interesting.

The city’s Chief Data Officer, Tom Schenk, spoke at Amazon Web Service’s DC Summit about OpenGrid and how it helps provide greater transparency and improves city operations.

City of Chicago is big user and producer of open source software, including for the R statistical suite. Watch one of the city’s data scientists gave a talk at the worldwide R user meeting on the city analytics projects.

Another month means another version of OpenGrid is available for the public. This month, the release includes some usability improvements and some technical API improvements. For developers, you can download OpenGrid v1.1.0 from the GitHub page.

AT&T named Chicago one of of the spotlight cities for AT&T’s smart city framework. As part of that, AT&T will be the wireless network used by the Array of Things to carry data from each sensor to the open data portal.

Image credits: “Taste of Chicago Blue Hour” by Justin Kern and licensed under Creative Commons Attribution-NonCommercial-NoDerivs 2.0 Generic (CC BY-NC-ND 2.0)

City of Chicago’s data science team is a big fan of open source software, including the venerable R statistical programming language. Attendees of the annual UseR! (use R) conference were able to see how we used R to predict where to send food inspectors in Chicago. Take a look at Gene Leynes, a data scientist for the City of Chicago, discuss the project at a lightening talk.


 

Banner image credit: The Hunt by Edsel Little and licensed under Creative Commons Attribution-ShareAlike Generic 2.0 (CC BY-SA 2.0)

This week, the City released the newest version of OpenGrid,  v1.1.0 with a number of enhancements that is now available in the project repository and users of opengrid.io.

Queries with Relative Dates

Now you can search and save queries where certain dates fall within relative date ranges.  A relative date range is a period of time that is relative to the current date.  OpenGrid supports the following relative date time periods:

  • Today
  • Yesterday
  • 1 minute ago, # minutes ago
  • 1 hour ago, # hours ago
  • 1 day ago, # days ago
  • 1 week ago, # weeks ago
  • 1 month ago, # months ago
  • 1 year ago, # years ago

For example, suppose you are a foodie and want to stay up-to-date on all of the hotspots that opened within the last week. You would select the Business Licenses Dataset, and set the License_description = “Retail Food Establishments.” Then apply another filter with the Issue Date between “1 week ago” and “today.”

relative-dates

For those using OpenGrid with user management, users can save queries with relative date. Running those queries at a later date will use the relative date, making it more useful for those who want to re-run queries within a moving time window.

Datasets alphabetized for easier discovery

Previously all of the datasets available in OpenGrid were not alphabetized, making the finding and selection of a dataset an arduous task.  Now you can view and access all of the datasets available to you in a more organized manner.

sorted-dataset

Improved Geospatial Filtering Performance on OpenGrid.io

At times, users would notice that queries using geospatial filters would not always return all the results and may omit some data. This issue has been fixed, though, opengrid.io is still limited to a maximum of 1,000 results for the time being.

API support of geospatial filtering

OpenGrid’s API now supports geospatial filtering calls. Previous versions of OpenGrid would search for all data before “filtering” for specific geographies. As a result, queries were extraneous and took longer than necessary. Now, queries can be provided which will only search for data within a given geospatial parameter.

Opengrid.io displays a maximum of 1,000 research results.  Previously, when a user would try to maximize their search results for a particular area via application of a geo-spatial filter, the filtering was happening on top of the 1,000 random records that were being pulled from Plenar.io.  Now that we’ve modified the API which uses geo-spatial filtering, we are able to support filtering on the service itself.  When a user applies a geo-spatial filter, the query will run on the defined area to find the top 1,000 results, as opposed to querying the entire database for the 1,000 results and then applying the geo-spatial filter.

Future of OpenGrid

Over the past few months, City staff has worked with Chicagoans from various neighborhoods to gather feedback on usability.  With the information we have gathered, the next phase of OpenGrid will be focused on making the data that is available in the app even more accessible and user friendly.  The City will also continue to collaborate with technologists who can help improve the platform to let residents explore open data, to help the city keep our streets safer, and to even make it useful for non-profits who may want to adopt the platform.

Interested in contributing to OpenGrid?

If you’re interested in collaborating with the city, you can take a look at the past meeting notes on the project’s Wiki. While the source code is online, the project’s documentation gives a concise overview for developers. Likewise, look at the instructions for contributors to help us run a useful, collaborative project.