In 2012, Mayor Rahm Emanuel signed an Executive Order which required the City of Chicago to have a data portal to post information about what the city does, how much it pays workers, and display other information that is useful for Chicagoans. Open data portals were still relatively new, but gaining in popularity. Chicago’s data portal went from a dozen data sets to hundreds. Since then, Chicago has learned more about what users need and which needed to be changed. Frequently, casual users would get lost in the ever-expanding data portal. While the amount of data increased, the interface did not and had not changed to accommodate the sheer volume of information.

Now, Chicago has launched a redesigned data portal based on this feedback. Visitors to the site will now see a clean interface that connects them with information, ranging from graphs, maps, to apps and raw data.

Desktop screenshot of the redesigned open data portal

Open data is not just for the tech savvy, but for all Chicagoans and visitors to the data portal. In fact, 40 percent of users describe themselves as someone in need of data or are just curious. Over the past 5 years, the Data Portal Team has receive a lot of feedback on the portal and a lot of questions. Those questions have given us insight to what is deficient and what is preventing more users from accessing it. In the past two years, Chicago has engaged in studies and focus groups to get feedback on a wide range of users.

The data portal now also connects users with other useful information, such as apps using Chicago data. Besides data itself, residents can use apps like SeeClickFix, Plow Tracker, SweepAround.us and other apps to get useful updates. Apps created by the City, associated non-profits or during city-sponsored hackathons will be included on the data portal, giving users access to something besides data on the portal.

Around 30% of Data Portal users were using some sort of mobile device like a smartphone or tablet. The new portal experience is optimized for people on mobile devices, not just desktops and laptops. The portal will be formatted to fit smaller screens with buttons that can be used with fingers. This is also important because research has shown that low-income Chicagoans are more likely to use their smartphone or tablet to access city services like the Data Portal. Improving the Portal on mobile devices also means greater equity of service to everyone.

Redesigned data portal on a tablet

Feedback goes beyond web surveys. We partnered with groups such as Smart Chicago Collaborative and ChiHackNight to conduct focus groups across users. In September 2016, the City of Chicago and Smart Chicago Collaborative hosted a feedback session on the new design at Woodson Regional Library in Washington Heights to give feedback on an earlier test version of the new portal.

Feedback from over a dozen casual, curious citizens was clear: they didn’t only expect to see data on the portal but also access to city services. So, the redesigned portal provides contextual information to services or information related to data. For instance, looking at the Business Licenses dataset also provides you links to apply for a business license using the city’s new online permitting system.

For the times you still cannot figure out how to use the portal, such as how to download data, Chicago has launched a dedicated YouTube playlist with tutorials on the data portal. Those tutorials are part of the Data Portal homepage so it’s easier to stumble across helpful tips.

Finally, the software developer, data scientists, startups and other tech savvy users obviously play a huge role in the redesigned data portal. There are clearer and more prominent links to the technical documentation needed to get started. Also, if you are interested in creating some software using the portal, the newly relaunched Chicago Developer’s website contains helpful documentation to get you started.

Designing a website is never “done”. You’ll see new tutorials added in the future and the roll out of even more new features in the near future. As always, the feedback we receive helps us plan the next set of features and data. Email or Tweet us with your feedback or questions.

 

Featured Image by Matthew Mazzei and released to the Public Domain (CC0).

Much as we take pride in our toughness through Midwestern winters, the real joy of living in this city is our Chicago summers.  High on the list of the summertime pleasures is outdoor dining at restaurants that offer it and finding such restaurants just got a bit easier.

Our new Sidewalk Cafe Permits dataset lists those businesses that have obtained the necessary permit to offer outdoor restaurant seating on the public way.  For those interested in long-term trends, the full dataset shows permits back to 2001.  However, if you just want to find a place to eat tonight, you will be more interested in the list of current permits.  Better yet, look at them on a map:

Permits are valid from March 1 to December 1 and more are added almost every day.  So, check back often and  bon appétit!

City of Chicago has been opening-up its technology and data so others can build amazing apps and services. Just like the people and companies who built a world-class architecture on city-build roads, Chicago open data and APIs let you can build world-class software on top of it. Now, residents have access to bus trackers, plow trackers, and new ways to report 311 requests on their phones and computer. These services make it easier to live and visit the city.

Redesigned dev.cityofchicago.org website on a laptop computer

A big part of this is ensuring software developers have access to the information and documentation needed to build these services. We have launched a completely redesigned dev.cityofchicago.org to help discover APIs and documentation in a central location. Now, there is a comprehensive place to find city-supported APIs and platforms that the City of Chicago and its sister agencies offer to the public. For instance, details on the Open311 API–which allows apps to create new 311 requests in city databases–is documented here.

Dev Blog

The site also includes a new blog aimed for software developers and data scientists. Dev Blog lets you know about upcoming changes that might impact existing apps and also give updates on outages or maintenance schedules. Unlike this blog, Dev Blog is targeted to engineers who need the technical details to keep their app updated and users happy.

As part of this, the Chicago Data Portal Status Blog has been retired and moved to the new Developer Blog. We’ll post any technical changes to the portal and let you know of outages as before. New posts will also be Tweeted from @ChicagoCDO.

Finally, the new website is open source, based on Jekyll. If you spot any problems with the website, such as type-o’s or other issues, we’d welcome your contributions on our GitHub site.

Feel free to reach out to us with questions or feedback through Twitter or email.

 

Featured image “O88/365: New Construction” © Don Harder 2017 and is licensed under Creative Commons Attribution-NoCommercial 2.0 Generic (CC BY-NC 2.0).

Although data portals offer a variety of tools to examine datasets, the ultimate “open” in “open data” is the ability to export the data to a file and do as one wishes with it.  However, this can get challenging with very large datasets, particularly over slower Internet connections.  It can also be inefficient if one needs only a portion of the data.

There are a number of techniques to attack this problem but a particularly good, yet frequently overlooked, one is to use the filtering at the core of a Data Lens page (often called a “Dashboard” on the Chicago Data Portal).

Using an example that comes up frequently in questions from our users, our Taxi Trips dataset contains over 107 million trips, as of this writing.  Downloading the full dataset in CSV format could take hours over even a fast connection and produce a very large file.

Suppose that you only needed the records from the last three months of 2016.  You may know that you can easily filter the Data Lens view to show those records.  Simply hover over the October 2016 slice on any of the time-based cards, click, drag across November and December, and release.  You have now selected approximately four million trips.

Image of a Data Lens card where October 2016 through December 2016 has been selected.

Using Data Lens to select three months of trips.

However, this is only the start.  Not only can you view those trips within the page — including how they affect other cards — but you can also export just those records.  To do so, click the Export button at the top of the page.  It will default to export all records but you can change that option to export only the currently selected records.

Screenshot of the Data Lens Export button, showing the option to download only selected records.

Data Lens Export button, showing the option to download only selected records.

You can even apply multiple filters.  If all you really need is trips from those three months originating in Logan Square, apply that filter, as well, on one of the map cards.

Screenshot showing filtering to just Logan Square pickups.

Filtering by Community Area

The download now becomes a very-manageable 24,963 records that download in a matter of minutes or seconds to a file under 10 MB in size.

As mentioned above, there are other ways to filter a large dataset.  For some more-complicated needs, it may be helpful to use them and they can be very powerful.  However, to make one or a few quick slices of a dataset and download the results, it is hard to beat the convenience of the graphical filters in a Data Lens page.

Feature image by luckey_sun and is licensed under Creative Commons Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0).

The City of Chicago has launched the newest Energy Benchmarking data as part of the Energy Benchmark ordinance passed in 2013, which includes information on energy consumption, size of buildings, ENERGY STAR® rating and more. For the first time, the data now includes residential buildings over 50,000 square feet. Nearly 2,700 properties are available in the data set, which was released alongside the most recent edition of the Chicago Energy Benchmarking Report for 2016.

Chicago Energy Benchmarking – 2015 Data Reported in 2016 – Map

Transparency on energy consumption data has always been a part of the Energy Benchmarking ordinance, where updates are published to the open data portal each year; allowing researchers, engineers, policymakers, and others to see and understand energy consumption of Chicago’s largest buildings. In 2014, the City released a 3D interactive energy map based on citywide energy consumption data as of 2010.

Read more about the Energy Benchmark ordinance, the findings, and data on its homepage.

Feature image by Jesse Collins and is licensed under Creative Commons Zero 1.0 Universal (CC0 1.0).

(c) 2014. Sean MacEntee. Licensed under CC-BY-2.0Did you notice it? Chicago’s main webpages were are safer and more secure than they were in 2016. Chicago’s homepage is now using “HTTPS” for its visitors. Things will not look different, but it provides a number of benefits for those visiting Chicago’s home pages. Now when visitors visit www.cityofchicago.org and select city websites, the connection will be secure so others cannot snoop or steal information being sent between users and the City of Chicago. Everything is encrypted between your device and the City of Chicago so it can even keep your information private when you are on a wireless connection.

City of Chicago websites will also be easier to find because sites using HTTPS are preferred by Google. In the future, encryption will allow webpages to load faster. New web standards, such as SPDY, are much faster and will only work with secure websites.

You may also notice that other websites you use, such as Facebook, Twitter, and Google also use HTTPS. Secure websites are becoming more important and now City of Chicago websites provide the same level of trust and privacy each time you visit the site. We have a few websites still being transitioned to HTTPS, just look for the padlock icon in your address bar.

OpenGrid was launched almost one year ago. Since that time, users have been providing feedback on how the platform can be even easier to navigate their neighborhoods. Over the past few months, the team has continued to focus on bettering the user experience and making data more easily accessible to the user as they navigate around the map. Users have told us that it took too many clicks to see data. For instance, users would need to click the “Find Data” to anytime they moved the map to get updated data.

The latest version of OpenGrid will now refresh each time the map is moved. You will no longer need to manually re-run your search when you want to see data for a new area. Each time the map moves or you zoom in or out, OpenGrid will instantaneously refresh with the newest data. Simply choose the data you want to see, click “Find Data” once, and you can explore Chicago.

The number of clicks needed to retrieve data has been significantly reduced in the last few months, making it easier to get data but still allowing powerful searching across over two dozen data sets.

OpenGrid in 2017

New features will be rolled out to OpenGrid team through 2017.  In the next few month we are aiming to add some new features:

  • Implement bubble plots on the map—The bubbles will vary in sized based on a numeric value associated with queried data. Consider, plotting the potholes in the city. It would be useful to be able to plot the site of the circle to correspond to the number of potholes filled on the block.
  • Display data with parent-child relationships
  • Graph time-series data
  • Improve the ability to share queries

Interested in contributing to OpenGrid?

If you’re interested in collaborating with the city, you can take a look at the past meeting notes on the project’s Wiki. While the source code is online, the project’s documentation gives a concise overview for developers. Likewise, look at the instructions for contributors to help us run a useful, collaborative project.

Chicago residents and visitors took more than 27 million taxi rides in 2015, traveling 83 million miles and spending more than $400 million.

The City of Chicago assures the quality and safety of those rides through its Department of Business Affairs & Consumer Protection (BACP).  We have long published the taxi drivers and vehicles licensed by BACP on the Chicago Data Portal.

As part of its mission, BACP is also authorized to collect information on taxi rides, themselves.  It does so through periodic reporting by two major payment processors believed to cover most taxis in Chicago.  Based on these reports, we are now able to provide a dataset of over 100 million Chicago taxi rides, dating back to 2013.

Chart of rides per month

Rides Per Month

More…

Back in April, the City of Chicago partnered with one of the Smart Chicago Collaborative anchor programs, the Civic User Testing Group (CUTGroup), to host a UX testing feedback session attended by more than 20 Chicago residents for OpenGrid. The purpose of these tests was not only to engage all of Chicago in the civic app development process, but provide valuable information to OpenGrid developers seeking to enhance the apps’ usability and performance. The newest version contains improvements to OpenGrid to make it easier to use. The latest release contains friendlier language, an improved user interface to highlight more important features while deemphasizing more technical options, and reducing the number of mouse clicks to see data. Below are some of the highlights of this release.

Simpler Advanced Search Panel

More Accessible Language throughout the application

During the CUTGroup testing, we saw that there were some opportunities to make the language more accessible. Users commented that the language throughout the application seemed to be targeted more towards developers and people with tech backgrounds. For example, in the Advanced Search panel, language such as “Geo-spatial filters” and “Queries” seemed to cause some hesitation to users.  Therefore, we simplified the language to make it more user friendly. “Advanced Search” was changed to “Find Data,” “Datasets and Standard Filters” was changed to “Select Data,” and “Submit” and “Reset” were changed to “Get Data” and “Clear Search” respectively.

Submit and Reset buttons are now set as footer

Previously, users would have to scroll to the bottom of the advanced search panel to run or clear a search. Now these buttons are static at the bottom of the page for easier accessibility.

before-and-after

Color coded dots for each dataset

When multiple datasets are added to a search, the dots for each dataset are now assigned a different color by default. Now that the user doesn’t have to manually assign a new color, we have cleaned up the advanced search panel to retract the color and opacity under “Color Options.”

color-options

Advanced Search Panel Retracts After Executing Query

To maximize the amount of data viewable to the user upon executing a search, the advanced search panel now retracts upon performing a search.

The ability to perform searches using OR

When applying filters for a dataset, users were only limited to applying an “AND” operator, now users can select “OR.”

Easier-to-read Map

Map Icons More Accessible

Users no longer have to retract the Find Data Panel in order to apply a layer or create a measurement on the grid. The layers and measurement icons have been moved to the boot left of the map for easier access.

map-icons

More Prominent Grid View

As seen in the screenshots above, we have changed the grid view access bar to a bold and black color, whereas previously it was gray and overlooked by users.

Map Extent by Default

When performing a search, all of the data is limited to a default geographical area shown on the map (i.e. map extent). The map extent is now setup to return results of the area that is currently visible on the grid. For example, if a user pans or zooms in to the Loop or Humboldt Park area and runs a quick search on crimes or fires, the results will return within the area that the user extended on the map.

Note: The default map extent does not apply when searching for points of interest like Starbucks, McDonalds, etc.

New Base Map

The new base map is now lighter and has a better contrast quality, allowing users to see the data displayed on the grid more clearly.

Map Legend

The map legend displays all of the datasets that were included in the users search. It appears at the bottom right of the grid when an advanced or quick search is executed.

legend

Future of OpenGrid

To continue our efforts in improving the user experience, the team is working towards making the map more responsive as users move around the city. As the user drags the map, the query will refresh to the new map extent. The team is also working towards upgrading Leaflet to v1.0 (previously v0.7) as well as all dependent plugins. Lastly, we will also be improving the Quick Search feature to allow users to perform multiple quick searches as well as perform a quick search along with an advanced search (now called, “Find Data”).

Interested in contributing to OpenGrid?

If you’re interested in collaborating with the city, you can take a look at the past meeting notes on the project’s Wiki. While the source code is online, the project’s documentation gives a concise overview for developers. Likewise, look at the instructions for contributors to help us run a useful, collaborative project.