City of Chicago’s Department of Innovation and Technology is hiring a pair of key positions that will shape the future of how the city uses technology in the future.

The Enterprise Architect is a senior role that oversees the city’s technical infrastructure, including network, server infrastructure, and more. They will play a key role in designing the future of the city’s architecture, including an increasing footprint in cloud computing services.

The city is also hiring a Postgres Database Administrator. The technology team has had a deep commitment to using open source software and will be expanding the use of Postgres as the enterprise database solution for city systems. The position will work with on-premise and cloud-based Postgres databases and helping migration some from Oracle to Postgres. You’ll work with a team of highly-experienced DBAs and reshape the city’s database platform.

On June 1 and June 2, 2017 OpenGrid will be featured as a part of Mozilla’s Global Sprint, a fun, fast-paced and two-day collaborative event to hack and build projects for a healthy Internet. Participants will be able to partner with a diverse network of scientists, educators, artists, engineers and others in person and online to innovate in the open.

Map of mozsprint 2017 location

mozsprint 2017 will be hosted June 1 and 2 at Make Offices, 1 North State Street #1500

OpenGrid is an open-source site that provides residents access to data about their community in a single map. Residents, researchers, and non-profits can use the data to find historical information on crimes, business licenses, building permits to real-time information on their phones or desktop. It’s mission is to improve Chicagoans’ ability to meaningfully use open data beyond the capacity of current data portals by providing residents and City leaders a way to visually understand complex municipal data. The site was launched by the City of Chicago in February 2016 and since then, released 4 new versions based on feedback we’ve received from casual users. Being an open-sourced project, the code is available to everyone, allowing this application to be made for residents and by residents.

Click button to register

If you are in the Chicago area and would like to attend in person please visit this site here for more details. Check-out OpenGrid’s roadmap and outstanding issues for ideas. Looking forward to collaborating with you!

Chicago has rolled-out a public test for a new feature designed to help scientists, journalists, archivists, and others to archive and preserve open data. In light of recent concerns about the removal of government data from public websites and data portals, particularly related to climate change, this new feature allows for data to be downloaded archived with a simple command in the R language.

All of the data, including maps, saved to your hard drive and time-stamped so you can easily see when the data was last preserved. The data are also compressed on your hard drive to to save space. In the near future, we will offer the ability to save data to the cloud to make it easier to save very large open data sets. Read the post on the city’s developers blog for instructions on installing and working with the the new features.

RSocrata is available for the R programming language and data hosted on Socrata open data portals, which is the largest open data portal provider in the world and is the portal for hundreds of federal agencies, state governments, and cities.

Right now, this is a public test of the new feature and may have some bugs. We welcome feedback and encourage you to report any issues on our bug tracker or email developers@cityofchicago.org.

RSocrata was developed and released by the City of Chicago in 2013 as a free, open source program. It has allowed researchers and programmers who use R to easily retrieve individual datasets. The program has been used to forecast ambulance and fire truck incidents, used by the City of New Orleans to generate performance reports, and mentioned in books teaching the fundamentals of data science. It is also used by Chicago’s data team on a regular basis, ranging from uploading and refreshing the open data portal to predicting food inspections.

 

Featured image “Open Data” Copyright (c) 2015 by Descrier

On Chicago’s Data Portal, there is no data more popular than the “Current Employee Names, Salaries, and Position Titles“. It is not as large or voluminous as many of the other data on the Data Portal, but has garnered a lot of interest. Now, the portal has even more information about city wages at your fingertips.

In addition to names, titles, and wages, you can now see if employees are full- or part-time and if they are paid on an hourly or annual salary. Even for hourly employees, whether they are expected to work 40, 35, or just 20-hours a week is viewable online and can be downloaded for free. A new dashboard also lets you explore the data interactively, quickly choosing and filtering for data you’d like to see, such as salaries by department or just for full-time workers.

New dashboard of current city employees lets you explore wages by department and employment level.

The new information on the Data Portal is designed to provide a more accurate picture of wages for city employees to help people look-up typical salaries, conduct research, or learn more about city wages.

Featured image “Chicago City Hall” © 2012 by Mike is licensed under Creative Commons Attribution-NonCommercial-NoDerivs 2.0 Generic (CC BY-ND-ND 2.0)

In 2012, Mayor Rahm Emanuel signed an Executive Order which required the City of Chicago to have a data portal to post information about what the city does, how much it pays workers, and display other information that is useful for Chicagoans. Open data portals were still relatively new, but gaining in popularity. Chicago’s data portal went from a dozen data sets to hundreds. Since then, Chicago has learned more about what users need and which needed to be changed. Frequently, casual users would get lost in the ever-expanding data portal. While the amount of data increased, the interface did not and had not changed to accommodate the sheer volume of information.

Now, Chicago has launched a redesigned data portal based on this feedback. Visitors to the site will now see a clean interface that connects them with information, ranging from graphs, maps, to apps and raw data.

Desktop screenshot of the redesigned open data portal

Open data is not just for the tech savvy, but for all Chicagoans and visitors to the data portal. In fact, 40 percent of users describe themselves as someone in need of data or are just curious. Over the past 5 years, the Data Portal Team has receive a lot of feedback on the portal and a lot of questions. Those questions have given us insight to what is deficient and what is preventing more users from accessing it. In the past two years, Chicago has engaged in studies and focus groups to get feedback on a wide range of users.

The data portal now also connects users with other useful information, such as apps using Chicago data. Besides data itself, residents can use apps like SeeClickFix, Plow Tracker, SweepAround.us and other apps to get useful updates. Apps created by the City, associated non-profits or during city-sponsored hackathons will be included on the data portal, giving users access to something besides data on the portal.

Around 30% of Data Portal users were using some sort of mobile device like a smartphone or tablet. The new portal experience is optimized for people on mobile devices, not just desktops and laptops. The portal will be formatted to fit smaller screens with buttons that can be used with fingers. This is also important because research has shown that low-income Chicagoans are more likely to use their smartphone or tablet to access city services like the Data Portal. Improving the Portal on mobile devices also means greater equity of service to everyone.

Redesigned data portal on a tablet

Feedback goes beyond web surveys. We partnered with groups such as Smart Chicago Collaborative and ChiHackNight to conduct focus groups across users. In September 2016, the City of Chicago and Smart Chicago Collaborative hosted a feedback session on the new design at Woodson Regional Library in Washington Heights to give feedback on an earlier test version of the new portal.

Feedback from over a dozen casual, curious citizens was clear: they didn’t only expect to see data on the portal but also access to city services. So, the redesigned portal provides contextual information to services or information related to data. For instance, looking at the Business Licenses dataset also provides you links to apply for a business license using the city’s new online permitting system.

For the times you still cannot figure out how to use the portal, such as how to download data, Chicago has launched a dedicated YouTube playlist with tutorials on the data portal. Those tutorials are part of the Data Portal homepage so it’s easier to stumble across helpful tips.

Finally, the software developer, data scientists, startups and other tech savvy users obviously play a huge role in the redesigned data portal. There are clearer and more prominent links to the technical documentation needed to get started. Also, if you are interested in creating some software using the portal, the newly relaunched Chicago Developer’s website contains helpful documentation to get you started.

Designing a website is never “done”. You’ll see new tutorials added in the future and the roll out of even more new features in the near future. As always, the feedback we receive helps us plan the next set of features and data. Email or Tweet us with your feedback or questions.

 

Featured Image by Matthew Mazzei and released to the Public Domain (CC0).

Much as we take pride in our toughness through Midwestern winters, the real joy of living in this city is our Chicago summers.  High on the list of the summertime pleasures is outdoor dining at restaurants that offer it and finding such restaurants just got a bit easier.

Our new Sidewalk Cafe Permits dataset lists those businesses that have obtained the necessary permit to offer outdoor restaurant seating on the public way.  For those interested in long-term trends, the full dataset shows permits back to 2001.  However, if you just want to find a place to eat tonight, you will be more interested in the list of current permits.  Better yet, look at them on a map:

Permits are valid from March 1 to December 1 and more are added almost every day.  So, check back often and  bon appétit!

City of Chicago has been opening-up its technology and data so others can build amazing apps and services. Just like the people and companies who built a world-class architecture on city-build roads, Chicago open data and APIs let you can build world-class software on top of it. Now, residents have access to bus trackers, plow trackers, and new ways to report 311 requests on their phones and computer. These services make it easier to live and visit the city.

Redesigned dev.cityofchicago.org website on a laptop computer

A big part of this is ensuring software developers have access to the information and documentation needed to build these services. We have launched a completely redesigned dev.cityofchicago.org to help discover APIs and documentation in a central location. Now, there is a comprehensive place to find city-supported APIs and platforms that the City of Chicago and its sister agencies offer to the public. For instance, details on the Open311 API–which allows apps to create new 311 requests in city databases–is documented here.

Dev Blog

The site also includes a new blog aimed for software developers and data scientists. Dev Blog lets you know about upcoming changes that might impact existing apps and also give updates on outages or maintenance schedules. Unlike this blog, Dev Blog is targeted to engineers who need the technical details to keep their app updated and users happy.

As part of this, the Chicago Data Portal Status Blog has been retired and moved to the new Developer Blog. We’ll post any technical changes to the portal and let you know of outages as before. New posts will also be Tweeted from @ChicagoCDO.

Finally, the new website is open source, based on Jekyll. If you spot any problems with the website, such as type-o’s or other issues, we’d welcome your contributions on our GitHub site.

Feel free to reach out to us with questions or feedback through Twitter or email.

 

Featured image “O88/365: New Construction” © Don Harder 2017 and is licensed under Creative Commons Attribution-NoCommercial 2.0 Generic (CC BY-NC 2.0).

Although data portals offer a variety of tools to examine datasets, the ultimate “open” in “open data” is the ability to export the data to a file and do as one wishes with it.  However, this can get challenging with very large datasets, particularly over slower Internet connections.  It can also be inefficient if one needs only a portion of the data.

There are a number of techniques to attack this problem but a particularly good, yet frequently overlooked, one is to use the filtering at the core of a Data Lens page (often called a “Dashboard” on the Chicago Data Portal).

Using an example that comes up frequently in questions from our users, our Taxi Trips dataset contains over 107 million trips, as of this writing.  Downloading the full dataset in CSV format could take hours over even a fast connection and produce a very large file.

Suppose that you only needed the records from the last three months of 2016.  You may know that you can easily filter the Data Lens view to show those records.  Simply hover over the October 2016 slice on any of the time-based cards, click, drag across November and December, and release.  You have now selected approximately four million trips.

Image of a Data Lens card where October 2016 through December 2016 has been selected.

Using Data Lens to select three months of trips.

However, this is only the start.  Not only can you view those trips within the page — including how they affect other cards — but you can also export just those records.  To do so, click the Export button at the top of the page.  It will default to export all records but you can change that option to export only the currently selected records.

Screenshot of the Data Lens Export button, showing the option to download only selected records.

Data Lens Export button, showing the option to download only selected records.

You can even apply multiple filters.  If all you really need is trips from those three months originating in Logan Square, apply that filter, as well, on one of the map cards.

Screenshot showing filtering to just Logan Square pickups.

Filtering by Community Area

The download now becomes a very-manageable 24,963 records that download in a matter of minutes or seconds to a file under 10 MB in size.

As mentioned above, there are other ways to filter a large dataset.  For some more-complicated needs, it may be helpful to use them and they can be very powerful.  However, to make one or a few quick slices of a dataset and download the results, it is hard to beat the convenience of the graphical filters in a Data Lens page.

Feature image by luckey_sun and is licensed under Creative Commons Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0).

The City of Chicago has launched the newest Energy Benchmarking data as part of the Energy Benchmark ordinance passed in 2013, which includes information on energy consumption, size of buildings, ENERGY STAR® rating and more. For the first time, the data now includes residential buildings over 50,000 square feet. Nearly 2,700 properties are available in the data set, which was released alongside the most recent edition of the Chicago Energy Benchmarking Report for 2016.

Chicago Energy Benchmarking – 2015 Data Reported in 2016 – Map

Transparency on energy consumption data has always been a part of the Energy Benchmarking ordinance, where updates are published to the open data portal each year; allowing researchers, engineers, policymakers, and others to see and understand energy consumption of Chicago’s largest buildings. In 2014, the City released a 3D interactive energy map based on citywide energy consumption data as of 2010.

Read more about the Energy Benchmark ordinance, the findings, and data on its homepage.

Feature image by Jesse Collins and is licensed under Creative Commons Zero 1.0 Universal (CC0 1.0).

(c) 2014. Sean MacEntee. Licensed under CC-BY-2.0Did you notice it? Chicago’s main webpages were are safer and more secure than they were in 2016. Chicago’s homepage is now using “HTTPS” for its visitors. Things will not look different, but it provides a number of benefits for those visiting Chicago’s home pages. Now when visitors visit www.cityofchicago.org and select city websites, the connection will be secure so others cannot snoop or steal information being sent between users and the City of Chicago. Everything is encrypted between your device and the City of Chicago so it can even keep your information private when you are on a wireless connection.

City of Chicago websites will also be easier to find because sites using HTTPS are preferred by Google. In the future, encryption will allow webpages to load faster. New web standards, such as SPDY, are much faster and will only work with secure websites.

You may also notice that other websites you use, such as Facebook, Twitter, and Google also use HTTPS. Secure websites are becoming more important and now City of Chicago websites provide the same level of trust and privacy each time you visit the site. We have a few websites still being transitioned to HTTPS, just look for the padlock icon in your address bar.