Polishing Minneapolis’ Wireless Civic Garden

closeThis post was published 7 years 11 months 13 days ago. A number of changes have been made to the site since then, so please contact me if anything is broken or seems wrong.

I’ve done some playing around with the citywide Wi-Fi here in Minneapolis, and I must say that the range of information accessible through the Civic Garden feature (which allows even non-subscribers access to City-related sites) is impressive.

However, while I understand that the whitelist of “free” domains is limited to noncommercial properties, there are a few exceptions that should be made. Or at least, some resources should be hosted by the City or proxied for Civic Garden users.

Metro Transit’s site

Visiting MetroTransit.org when online via the Civic Garden is a little weird. The home page is a lot longer than usual—actually, most pages are longer than usual—due to the absence of JavaScript libraries hosted at ajax.googleapis.com.

Because of the missing code, features that normally hide away in compact accordion stacks or appear when the mouse is moved over them are left in the open. One of them even steals focus when the page has loaded, making the view jump most of the way down the page. It took me a while to figure out why the page was scrolling by itself.

The navigation is broken for all but the top-level sections, because the missing code runs the drop-down menus that allow deeper browsing into the site. On the front page, a series of five images depicting the various Metro Transit services1 that is normally an automatic slideshow with mouse interaction expands to five panes stacked down the page—and the links embedded in them don’t work.

On the right side of the page, a clutter of tools appears where there is normally a neat stack of expandable options. One of them is the culprit for the page-scrolling I observed, and it gets annoying after a few pageviews to have to scroll to the top of each new page loaded. (Somewhere in the page’s code, a JavaScript snippet that doesn’t rely on one of the missing libraries is placing the caret2 in a text input near the bottom of the page, and most browsers automatically scroll to make such a “focused” element visible.

Glancing at the page’s source, I notice immediately which files must be the problem. Two <script> tags include the jQuery and jQuery UI libraries from Google’s CDN. This practice usually improves speed, since the likelihood of the files already being cached by a visitor’s browser is increasing as more and more sites start using these Google-hosted versions of popular JavaScript libraries instead of their own copies—but in this case, it’s causing breakage for a subset of users. Google’s service is not whitelisted as part of the Civic Garden.

Solutions

Two solutions present themselves, and they are both simple.

Ideally, Metro Transit would pass a request up the chain for ajax.googleapis.com to be whitelisted. Not only would doing so solve the problem for their site, but it would also allow any other Civic Garden website to take advantage of Google-hosted libraries without any further work from either Civic Garden administrators or individual site maintainers.

This first solution also has the potential to save bandwidth usage, since Google sends aggressive caching instructions along with the files hosted on its CDN. More Civic Garden sites using libraries hosted by Google would result in negligible increases in data transfer, because the same files would be downloaded once and then cached for use by any site requesting them. Saving bandwidth on the free Civic Garden would open up more of the pipe for paying subscribers—an outcome with which U.S. Internet would no doubt be pleased.

Alternatively, Metro Transit could add the core jQuery and jQuery UI files to the pre-existing /ClientScript/ directory, which I can see already contains plugins to those libraries, the Cufón library,3 and a font file for Cufón to use, among other things.

This alternate solution is a good fallback if the higher powers in control of the whitelist refuse a request to allow access to ajax.googleapis.com. It only solves the problem for Metro Transit’s website, but it would fix the issues discussed above.

A third, much more complicated, option is described below. Obviously, if it were applied to the Metro Transit problem, ajax.googleapis.com would be used where www.google.com is in those examples. While it would also work, it is unnecessarily complicated for the scope of the problem facing Metro Transit’s website, and that is why I don’t count it as a solution here.

Resolution

Some time ago I contacted Metro Transit using the feedback form on their site to notify them about the breakage and propose (in brief) my solutions. I received a response just a few days ago, with the welcome news that they will be fixing the problem in the next site update by hosting the JavaScript files themselves. Not the ideal solution, but definitely the easier of the two possibilities I could think of.

Way to go, Metro Transit! You’ve beaten me to the punch. Not that it’s hard to do these days, what with my posting frequency and all… 😉

The City’s site

Located at www.ci.minneapolis.mn.us, the Official Website of the City of Minneapolis has a wealth of information on everything from regulations to recycling and more. It allows access to City Council agendas, a list of what can and cannot be left for the recycling program, and countless other unexciting but eminently useful bits of information.

The main problem with the City’s site as viewed through the Civic Garden access is that it is impossible to search. Submitting a query through the search box at the top of any page leads the user to a page that says “Search the Minneapolis Web Site” above another (empty) search box. And pretty much ends there.

It’s great to see that the City (or at least its Web developers) is embracing modern Web services like Google’s Custom Search Engine, but all the resources required to fetch and display results come from www.google.com, a domain blocked when using Civic Garden access.

Not an Easy Problem

Solving this problem is a bit more difficult. Whitelisting www.google.com is out of the question, as that would also allow free access to many of Google’s consumer-oriented services including its trademark search engine, calendar, feed reader, and so on.4 Unfortunately there are no easy solutions here. Hosting the JavaScript files doesn’t solve the problem because those files in turn load other files whose locations are embedded in the code.

Implementing some sort of proxy would seem to be a solution, but there’s still the matter of hard-coded resource locations. Nothing returned by Google would request files via a City-controlled proxy, no matter how sophisticated the proxy.

There’s also the matter of load. Obviously any solution involving the use of City hosting services should be restricted to those users who need it—that is, Civic Garden users—to avoid unnecessary load on the servers. But there might not be a way of separating the “needs” from the rest of the crowd in a way that would allow the server to send different pages to those who need them.

Best Idea Forward

Without knowing more about the network architecture, I can come up with only one possible solution.

The flow would go something like this:

  1. User loads search page, and browser requests resources from Google
  2. U.S. Internet network5 receives and recognizes requests destined for www.google.com
  3. Network scans a list of allowed request patterns to www.google.com; such a list allows only the resources needed for Google Custom Search
  4. User’s browser receives the needed resources
  5. Google’s Custom Search code sends its requests to retrieve results, which are filtered through the same mechanism at the network level and allowed to return data to the user, completing the search

It’s a rough description, but generally all that’s needed is an extension of the domain-based filtering to enable filtering on request patterns—that is, the contents of the GET line in the request headers.

If the requested hostname matches www.google.com, that request is sent to a second filtering routine that performs pattern analysis (via regular expressions or what-have-you) on the requested path. /jsapi and /coop/cse/* can get through and return those resources to the user; /reader/view/ and /webhp?q=denied can’t, and redirect to the subscription login page (the current behavior for all non–Civic Garden sites).

Implementing this solution would require analysis of all the possible requests generated by Google Custom Search, though Google might have available (or be willing to provide) a reference of how Custom Search works. Once put in place the filtering expansion would enable any site in the Civic Garden to use the service and have it work for everyone, without changing anything else. It might also require changes to the network equipment that runs the citywide wireless service, but such upgrades would prove useful in short order as more City services were made available to Civic Garden users thanks to the accessibility of search. (See next section)

Other Applications

While the main problem with the City’s site as accessed via the Civic Garden is the lack of search, there are other issues.

Forms, for instance, seem to mostly be hosted on external sites that are not included in the Garden whitelist. Much information is given about the services these forms can be used to obtain (such as snow emergency notifications by telephone or email), but filling out the forms is impossible.

A complete audit of all external resources called by the City’s site (and in general, all Civic Garden sites) could provide a list of domain names and resource paths for whitelisting. The above-described filtering system could be extended with the contents of such a list so specific pages from commercial sites used on City properties could be made available, while still blocking effectively all commercial traffic from the Civic Garden.

Enabling access to third-party resources that are currently blocked, despite being included in Civic Garden properties, would provide an even greater return on the investments of time and (possibly) money in the upgrades of network hardware and firmware that would likely be necessary to support such a filtering system.

I emailed the City about this and was notified several days later that my message had been forwarded to their IT department. At that time I hadn’t come up with this new filtering idea, so I’ve contacted them again with a link to this post. Maybe they’ll read it, maybe not; but it’s been a nice thought experiment.


Notes:

  1. Which are: Bus, light rail, Northstar commuter train, bicycle accommodations, and Rideshare (car or van pools). []
  2. caret: the blinking line or box often used to enter text on a computer []
  3. Cufón replaces specified text elements with graphics rendered dynamically by the browser to provide more control over typography than the current lowest-common-denominator browser-native technologies. []
  4. I for one would love it if the City had Google services whitelisted so I could check my email and calendar from pretty much anywhere for free, but I can understand the need to block commercial sites on a publicly funded network. []
  5. U.S. Internet is the local ISP that was awarded the contract to build and run the citywide wireless service. []

dgw

I am an avid technology and software user, in addition to being reasonably well-versed in CSS, JavaScript, HTML, PHP, Python, and (though it still scares me) Perl. Aside from my technological tendencies, I am also a theatre technician, sound designer, violinist, singer, and actor.

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail (or subscribe without commenting)

Comments are subject to moderation, and are licensed for display in perpetuity once posted. Learn more.