Chicago’s Chief Information Officer, Brenna Berman, offered her insights on the future city in a recent special edition of Internet of Things (IoT) in CIO Review. Read how IoT can reshape city operations, deliver real-time information, and provide more information for policy.
Back in April, the City of Chicago partnered with one of the Smart Chicago Collaborative anchor programs, the Civic User Testing Group (CUTGroup), to host a UX testing feedback session attended by more than 20 Chicago residents for OpenGrid. The purpose of these tests was not only to engage all of Chicago in the civic app development process, but provide valuable information to OpenGrid developers seeking to enhance the apps’ usability and performance. The newest version contains improvements to OpenGrid to make it easier to use. The latest release contains friendlier language, an improved user interface to highlight more important features while deemphasizing more technical options, and reducing the number of mouse clicks to see data. Below are some of the highlights of this release.
Simpler Advanced Search Panel
More Accessible Language throughout the application
During the CUTGroup testing, we saw that there were some opportunities to make the language more accessible. Users commented that the language throughout the application seemed to be targeted more towards developers and people with tech backgrounds. For example, in the Advanced Search panel, language such as “Geo-spatial filters” and “Queries” seemed to cause some hesitation to users. Therefore, we simplified the language to make it more user friendly. “Advanced Search” was changed to “Find Data,” “Datasets and Standard Filters” was changed to “Select Data,” and “Submit” and “Reset” were changed to “Get Data” and “Clear Search” respectively.
Submit and Reset buttons are now set as footer
Previously, users would have to scroll to the bottom of the advanced search panel to run or clear a search. Now these buttons are static at the bottom of the page for easier accessibility.
Color coded dots for each dataset
When multiple datasets are added to a search, the dots for each dataset are now assigned a different color by default. Now that the user doesn’t have to manually assign a new color, we have cleaned up the advanced search panel to retract the color and opacity under “Color Options.”
Advanced Search Panel Retracts After Executing Query
To maximize the amount of data viewable to the user upon executing a search, the advanced search panel now retracts upon performing a search.
The ability to perform searches using OR
When applying filters for a dataset, users were only limited to applying an “AND” operator, now users can select “OR.”
Map Icons More Accessible
Users no longer have to retract the Find Data Panel in order to apply a layer or create a measurement on the grid. The layers and measurement icons have been moved to the boot left of the map for easier access.
More Prominent Grid View
As seen in the screenshots above, we have changed the grid view access bar to a bold and black color, whereas previously it was gray and overlooked by users.
Map Extent by Default
When performing a search, all of the data is limited to a default geographical area shown on the map (i.e. map extent). The map extent is now setup to return results of the area that is currently visible on the grid. For example, if a user pans or zooms in to the Loop or Humboldt Park area and runs a quick search on crimes or fires, the results will return within the area that the user extended on the map.
Note: The default map extent does not apply when searching for points of interest like Starbucks, McDonalds, etc.
New Base Map
The new base map is now lighter and has a better contrast quality, allowing users to see the data displayed on the grid more clearly.
The map legend displays all of the datasets that were included in the users search. It appears at the bottom right of the grid when an advanced or quick search is executed.
Future of OpenGrid
To continue our efforts in improving the user experience, the team is working towards making the map more responsive as users move around the city. As the user drags the map, the query will refresh to the new map extent. The team is also working towards upgrading Leaflet to v1.0 (previously v0.7) as well as all dependent plugins. Lastly, we will also be improving the Quick Search feature to allow users to perform multiple quick searches as well as perform a quick search along with an advanced search (now called, “Find Data”).
Interested in contributing to OpenGrid?
If you’re interested in collaborating with the city, you can take a look at the past meeting notes on the project’s Wiki. While the source code is online, the project’s documentation gives a concise overview for developers. Likewise, look at the instructions for contributors to help us run a useful, collaborative project.
Earlier this month, Mayor Emanuel and Mayor of London, Sadiq Khan visited toured Chicago’s tech centers, including 1871 and UI Labs to look at Chicago’s startup culture and data-driven innovation. Chicago and London signed a data alliance for “bi-lateral relationship to enable the exchange of knowledge and expertise with the intention of driving expanded growth and opportunity for all people in the two cities.” The two cities will sharing solutions to avoid duplication and share ideas on how to use data within large cities.
Pleased to sign new tech agreement with @chicagosmayor. As Mayor, I’ll take London’s tech sector to the next level.
— Sadiq Khan (@SadiqKhan) September 16, 2016
White House highlights Chicago data and technology initiatives
- Array of Things are being installed across Chicago to collect environmental, noise, density and other data that is designed to improve the city and is limited to protect individual privacy. The data will be made publicly available online. You can now follow along when new nodes are installed on the open data portal. See a map of new locations.
- Digging for the next innovation! An underground utility is struck every 60 seconds in the United States, causing billions in damage. Chicago partnered with UI Labs’ City Digital and a consortium to build and pilot new technology to scan and map underground infrastructure. Take a look at the video to show how this information can be browsed in the future.
- The White House highlighted Array of Things and the Underground Mapping project as examples of innovative programs as part of their $80 million expansion of investments in the Smart Cities Initiative.
Redesigned websites ready for public feedback
- Chicago launched a pilot of the next open data portal and is looking for feedback from users. Test it and drop a line to firstname.lastname@example.org.
- The beta of the data portal was tested with groups at ChiHackNight and Smart Chicago’s Civic User Testing (CUT) Group. Both tech-savvy and casual users piloted the site and provided their feedback directly to the open data team, who will be adjusting the site for a better experience.
— Sonja Marziano (@ssmarziano) September 28, 2016
- Chicago’s Annual Financial Analysis report went digital for the first time this year. The report was assembled using open source and available tools, suck as Jekyll and GitHub, forgoing more costlier methods.
- Chicago has released a half-dozen APIs in the past 5 years. A beta of a new developer portal is now available to connect developers and researchers with documentation and updates to those services. You can reach out to email@example.com to provide your thoughts.
Open Data Portal continues to expand
- Dozens of new education-related data sets are on the open data portal. See maps of school locations for the past 10 years, school profiles, and much more.
- Business Activity codes — which describe what a business does — has been added to the open data portal’s Business Licenses data set.
Chicago is leveraging open data as part of it’s predictive analytics
Chicago’s Chief Data Officer spoke with Manhattan Institute’s Aaron Renn on the intersection of open data and predictive analytics in a recent podcast:
Banner image “Great Lakes in Sunglint” by NASA (c) 2012 and licensed under Creative Commons Attribution Non-Commercial 2.0 (CC BY-NC 2.0)
This Friday, Chicago’s open data team will be presenting a free workshop on how to use the city’s open data portal. The data portal contains a wide range of data, from liquor licenses, building permits, incidents of crimes, school profiles, and much more. You will see how you can search and find data related to your community and the ability to make maps and graphs using this freely accessible data. You don’t need to be an IT professional or a researcher to work with data from the portal and this workshop will help connect you with data about your school, business, and community.
Register for the event here. It’s free and open to the public.
If you cannot make the workshop, you can see other tutorials on the data portal from the team’s YouTube channel:
Ever wonder what’s beneath the roads in Chicago?
Chicago–founded in 1837–has a maze of water pipes, cables, and tunnels that have accumulated over the city’s 180 year history. There are communication lines, both old and new, that crisscross the streets. Every day in the city, construction crews conduct something akin to open heart surgery each day to upgrade water main lines, repair gas lines, and even lay new fiber optic cable while removing telegraph wires. Of course, this is not unique to Chicago. In the United States, over 400,000 underground excavations take place each day. Since some of this information is outdated or not easily accessible, there is an excavation incident roughly every 60 seconds and causing an estimated $1.7 billion in costs.
The City of Chicago has teamed-up with UI Labs’ City Digital to launch a pilot with a that will begin to deploy a platform to collect and create a 3-D map of underground infrastructure. This platform will provide a clearer insight so engineers and crews can better plan their work, identify issues, and lower the overall time to complete underground excavations and reduce costs. By housing this information in a platform, it will allow the City of Chicago to securely share limited data with utilities and other organizations that need to understand what is below Chicago’s roads, while limiting to only sections that are needed to be known.
Digital maps do already exist of Chicago’s subsurface labyrinth. But these often represent construction plans. Sometimes crews go beneath the street only to find there is a need to improvise, often because they come across undocumented structures that impede their initial plans. This new approach will use basic digital cameras and sophisticated software to scan the actual underground assets in the city.
A project like this cannot be done alone. City of Chicago teamed up with UI Labs’ City Digital and their partners, HBK Engineering and Accenture, to kick-start this project.
New technology developed by University of Illinois at Urbana-Champaign’s Real-Time and Automated Monitoring and Control Lab (RAAMAC) and Chicago start-up CityZenith will allow the city to scan and 3-D map Chicago’s underground assets.
One difficult part of a project like this is collecting the correct data that is economical and relies on widely available techniques. RAAMAC’s solution allows images captured from off-the-shelf cameras, such a basic DSLR camera.
Those images are inputted into RAAMAC’s software to create a stereoscopic image that renders a computer model of the underground infrastructure. Over a dozen images can be seemlessly captured during typical repair and construction adding no additional time for data collection. On-site engineers are able to collect this information without interfering with crews and with cameras that are affordable.
These renderings can be provided to engineers, planners, and crews. These models can also be securely shared between the City of Chicago and other companies with underground infrastructure to improve project planning and limiting accidents. Sharing can be limited to specific areas in the city so information does not get over-shared. For those with access, they will be able to see detailed renderings of the location of the underground infrastructure with rich metadata on each pipe, fiber optic connection, water main, and more:
The visualization will provide more than just a sophisticated interface. Having a full understanding of the city’s underground infrastructure will help avoid accidents that have a history of interrupting commutes or worse.
Chicago’s open data portal rapidly expanded when Mayor Emanuel’s administration took the reins in 2011. The data portal grew from a handful of data sets to several hundred data sets covering a range of topics. Since then, the portal has been viewed almost 40 million times and continues to grow. The data portal welcomed over 61,000 visitors last month, a 4 percent increase over the previous year. These users are have diverse backgrounds with approximately 65% of users are everyday residents, not data professionals or students. And almost a third of users are coming from mobile devices such as smartphones or tablets.
This week, the city has begun to pilot a new open data portal design. Based on the feedback from the past 5 years and looking at recent trends, the new design tackles three main goals:
- Improve searching and discovery to highlight interesting data sets and make it easier to find new ones
- A clean, modern interface that works for both desktop and mobile devices
- Connecting visitors to the Open Data Portal to other information, apps, and resources they find interesting or more useful than the raw data itself
New design and new features
A number of changes are being piloted. Most notable is the homepage where you’ll be greeted with a carousel of featured data sets, a large search box, and other relevant links. Search is key to a data portal so it is 16x larger than the current search box. The layout is also a cleaner, contemporary design that works on a desktop or mobile device. If you notice the site doesn’t work well on your device, operating system, or web browser, we strongly encourage you to reach out to us.
The search box is powered by a new search engine which will yield better, more relevant results. Like the homepage, the search results page works well on mobile devices so you can discover a useful data set regardless of your browser. The search results should be more relevant as well, which should result in less paging through results.
You can also look at a pilot of the new “data set landing pages” that display metadata about the data set. The open data team often receives questions about the definition of columns even when answers for those questions are included in the metadata. The current site, unfortunately, displays the first 20 or so rows of the data and suppresses metadata. The new landing page, however, displays the metadata, last updated data, and other information that answers many of the common questions of a data set.
The main website also contains links to relevant information. Recent posts from Digital Chicago, such as announcements of new data sets, will be readily available for users. Likewise, some users may be visiting the data portal but would rather see visualizations or user-friendly apps built on open data like OpenGrid or the city’s plow tracker. Finally, the portal includes a training video to give users an overview on how they can use the portal.*
The open data team will be at ChiHackNight this Tuesday to meet with their members to get feedback from some of the biggest power users who frequently code and create apps. This week the city will also be at Woodson Regional Library at 95th and Halsted with the Civic User Testing Group to get feedback from non-power users who often want to answer questions relevant to them. And, over the coming weeks, we seek your feedback as well.
When visiting the portal, simply click “Contact Us” and send any feedback, suggestions, or questions. Also feel free to reach out over Twitter to @ChicagoCDO.
One-stop-shop for developers
Chicago is also piloting a new developer portal. In the past 5 years, the City of Chicago and related agencies have released several new Application Programmer Interfaces (APIs) that let developers create apps based on city data. Besides the open data portal APIs, Chicago now have APIs that provide bus and train arrival times, allow you to submit 311 requests, or even report community concerns to the Chicago Police Department. The new developer portal is designed to connect you with those projects and documentation as a one-stop-shop for developers.
The developer portal also provides a blog that will replace the Open Data Portal Status Blog. Chicago started that blog to communicate technical changes and issues to the developer community. In order to provide a better one-stop-shop for users, that blog will redirect users to this new website.
The code for the new developer portal is open-source and on the city’s GitHub site. We would love to get your feedback on the issues page. This website will eventually replace the existing dev.cityofchicago.org, which was launched in 2011.
* The current tutorial is based on the old website design, but this will be updated in the future.
The first Array of Things sensor went online on September 1st and since then, 3 more sensors have gone online. Array of Things was deployed with a commitment to transparency and today we’ve launched an interactive map of the installed and planned sensors to be installed on Chicago’s Open Data Portal.
You can download the raw data behind the map from the portal as well.
Array of Things will continue to expand throughout the next couple of years for a total of 500 sensors. Later this year, after the data from the sensors are validated, data will be streamed online and in the open so researchers, citizen scientists, and the curious can analyze the data.
Fall is around the corner and so is school. Now there is a lot extra on the data portal for those who are interested in school data. Chicago Public Schools has posted 10 years of maps on the City of Chicago Open Data Portal, ranging from school locations, attendance boundaries, safe passage routes, Charter school “preference” and geographic network boundaries, and Local School Council (LSC) Voting Districts.
Each one of these data sets can be interactively viewed on the data portal or downloaded to make your own custom maps. The following datasets (50 altogether) are currently available on the City’s Open Data Portal:
- School Locations from 2006-07 to 2016-17
- Elementary School Attendance Boundaries from 2006-07 to 2016-17
- Middle School Attendance Boundaries from 2006-07 to 2016-17
- High School Attendance Boundaries from 2006-07 to 2016-17
- Charter School “Preference” Boundaries for 2014-15 and 2016-17
- Geographic Network Boundaries
- Safe Passage Routes for 2015-16 and 2016-17
- Local School Council (LSC) Voting Districts for 2016-17
These data compliment dozens of education data sets already on the Open Data Portal, including the location of Chicago Early Learning programs and annual progress report cards.
Details for the data nerds
There is a special treat for school data enthusiasts who want to combine and merge all of these data. Attribute table field names and data types (including test field lengths) will remain consistent for all attendance boundary and school location shapefiles. This means, for instance, that you can easily match schools listed in the Local School Council (LSC) with data in the School Locations, reducing the time and effort of trying to match school names.
There will be some exceptions. Prior to school year 2007-08, a 4-digit “UNIT_ID” format was used. For 2007-08 and on, the format switches to a 6-digit “SCHOOL_ID”. The “UNIT_ID” field has been retained for years 2007-08 through 2010-11, which can be used as crosswalk to link earlier years to later ones.
School names will vary slightly for a number of schools due to different naming conventions employed in previous years. It is best to use the “SCHOOL_ID” fields when linking/merging historical data. Some school addresses will also change as a result of relocations or co-locations with other schools over the years.
The school locations data set will also have some exceptions. School Type (SCH_TYPE) will vary slightly over the years. Generally, schools will be categorized as either an elementary, middle, high school (all District-operated), charter, contract, or alternative.
For more information, please email firstname.lastname@example.org.
With almost eleven years of historical data and more than one currently active license for every five City residents, Business Licenses is one of the most popular datasets on the Chicago Open Data Portal and a key resource for understanding economic activity. However, a known limitation has been that the License Code and License Description sometimes do not make it clear what the business actually does.
We have now been able to go deeper into the system the Department of Business Affairs & Consumer Protection uses to manage the tens of thousands of licenses issued or renewed every year to add Business Activities (and corresponding Business Activity IDs useful for quick filtering) to many license records. For example, the “Limited Business License” that makes up almost 40 percent of active licenses is, admittedly, not terribly descriptive. With the new Business Activities field, one can distinguish Hair Services, Tax Preparation, Car Washes, and the many other types of business activity that are recognizable to most people potentially shopping at these businesses.
It is important to note that some business licenses cover more than one type of Business Activity. In these cases, all available activities (and, again, corresponding codes) are listed in the columns, separated by the “pipe” character (|).
With the addition of this more-specific information on business activities of each license, we have removed addresses from some licenses where that information might unreasonably compromise privacy, currently fewer than 0.05 percent of all license records.
As a reminder of a longstanding feature that might be overlooked, the most useful records for many purposes are the currently active licenses (as opposed to expired ones). A filtered list of just these licenses is available as both a table and a map.
City of Chicago is continue to build an outstanding technical team and there are openings ranging from senior leadership, GIS, and database administration.
Working for the city provides a number of excellent benefits, such as:
- Make a direct impact on the 2.8 million residents and over 46 million visitors to Chicago.
- Modernize government processes for the 21st century, which has included paperless reform, driving the use of modern tools, and
- Understand how Chicago functions and drive changes to improve efficiency and quality of life.
- Excellent benefits
Take a look at the openings below and apply today.
- Director of Project Management Office – until filled
- GIS Analyst – available until September 16
- Principal Database Analyst – Oracle Apps / E-Business Suite – available until September 16