Visualizing Geospatial traffic data in Kepler.
Fast, elegant geospatial visualizations of hundreds of thousands of traffic data points using Kepler.gl and HERE traffic data
Skanda Vivek
Kepler.gl is Uber’s open source geospatial toolbox. Often, visualizing large-scale geospatial data sets can be quite challenging . There are many Python packages including Folium, GeoPandas, and Plotly that do a great job; but it can be a challenge to find the right plot backgrounds. It can also be computationally challenging on a local machine to plot thousands or hundreds of thousands of data points.
This is where Kepler.gl has a huge advantage. Kepler.gl is a browser based framework that loads CSV, JSON, or GeoJSON data easily and produces elegant visualizations with minimal formatting. In this example, I go through visualizing hundreds of thousands of traffic data points obtained from the HERE Traffic API in a few seconds.
Let’s get started!
Obtaining Traffic Data:
I’ve obtained traffic data from the HERE Technologies traffic flow API. I’ve written a few articles focused on how to extract traffic flow data:
from matplotlib import pyplot as plt
import numpy as np
import matplotlib.cm as cm
import requests
from bs4 import BeautifulSoup
import xml.etree.ElementTree as ET
from xml.etree.ElementTree import XML, fromstring, tostring
import pandas as pd
API_KEY='YOUR_API_KEY'
page = requests.get('https://traffic.api.here.com/traffic/6.2/flow.xml?app_id='+API_KEY+'&bbox=42.37,-71.20;42.22, -70.98&responseattributes=sh,fc')
file = open('speeds.txt', "w")
file.write(page.text)
file.close()
soup = BeautifulSoup(open('speeds.txt'))
roads = soup.find_all('fi')
a1=[]
loc_list_hv=[]
lats=[]
longs=[]
sus=[]
ffs=[]
c=0
for road in roads:
#for j in range(0,len(shps)):
myxml = fromstring(str(road))
fc=5
for child in myxml:
#print(child.tag, child.attrib)
if('fc' in child.attrib):
fc=int(child.attrib['fc'])
if('cn' in child.attrib):
cn=float(child.attrib['cn'])
if('su' in child.attrib):
su=float(child.attrib['su'])
if('ff' in child.attrib):
ff=float(child.attrib['ff'])
if((fc<=10) and (cn>=0.7)):
shps=road.find_all("shp")
for j in range(0,len(shps)):
latlong=shps[j].text.replace(',',' ').split()
#loc_list=[]
la=[]
lo=[]
su1=[]
ff1=[]
for i in range(0,int(len(latlong)/2)):
loc_list_hv.append([float(latlong[2*i]),float(latlong[2*i+1]),float(su),float(ff)])
la.append(float(latlong[2*i]))
lo.append(float(latlong[2*i+1]))
su1.append(float(su))
ff1.append(float(ff))
lats.append(la)
longs.append(lo)
sus.append(np.mean(su1))
ffs.append(np.mean(ff1))
Saving the Files
Kepler.gl likes files to be saved as CSV, with distinct columns. If you want an instantaneous geospatial plot, you need to label the columns as ‘latitude’ and ‘longitude,’ for spatial locations, and a third column for the color palate (which is speed in the case of traffic information).
lats_r=[]
longs_r=[]
speeds_r=[]
for i in range(0,len(lats)):
for j in range(len(lats[i])):
lats_r.append(lats[i][j])
longs_r.append(longs[i][j])
speeds_r.append(sus[i]/ffs[i])
print(i,len(lats))
data_bos=np.array([lats_r,longs_r,speeds_r])
df=pd.DataFrame(data=data_bos[:,1:].T,columns=["latitude", "longitude","speeds"])
speed2=np.array(df['speeds'])
w=np.where(df['speeds']<0.25)
speed2[w]=0
w=np.where((df['speeds']<0.5) & (df['speeds']>0.25))
speed2[w]=1
w=np.where((df['speeds']<0.75) & (df['speeds']>0.5))
speed2[w]=2
w=np.where(df['speeds']>0.75)
speed2[w]=3
df['speeds']=speed2
df.to_csv('df_speeds.csv')
Visualizing in Kepler.gl
Now that we have saved the data in the right format, we can drag and drop — it’s that simple!
Loading files in Kepler.gl
You automatically get a nice image that can be saved.
Exporting Images from Kepler.gl
However, when you look closely — the color scale is shown according to quantile — which depends on the data and what proportion of the data is at what speed. For example, if most of the traffic is fast moving (usually the case late at night or very early in the morning), the color palate will be skewed towards these higher speeds.
However, this is not what we want. We want colors to clearly indicate absolute speeds: just like in Google Maps traffic. In this case, we choose colors based on quantized value, rather than quantile, as below.
Choosing whether to color based on quantized value or quantile in Kepler.gl
There are also options to change how the symbols look as well as the color palate. In doing so, we can generate some pretty elegant visualizations — as in the header image of this article!
Modifying the colors and symbols in Kepler.gl
Conclusions
Kepler.gl is great for generating elegant visualizations with minimal effort, relatively quickly. Plotting hundreds of thousands of points on my local machine using OSMnx and other Python libraries takes much longer in comparison.
However, where Kepler.gl does a good job in terms of visualizations with minimal inputs, one also encounters the problem of minimal customizability.
For example there are basically only 3 backgrounds — dark, light, and satellite. There could be cases where other types of backgrounds are needed.
But Kepler.gl does a great job of covering the geospatial visualization essentials.
Upvote
Skanda Vivek
Senior Data Scientist in NLP. Creator of https://www.answerchatai.com/
Related Articles