Menu Sidebar
Menu

Raspberry Pi Kitchen Dashboard

2One of my goals for 2015 was to build and assemble a Raspberry Pi based Kitchen Dashboard. The original idea was to use Dashing.io via this tutorial, but the Ruby requirement of Dashing.io made developing custom widgets tougher for me on Windows, so I ended up switching to Atlasboard.

Challenges

There were many reasons why this project had core problems with the idea. Some of those included:

  • Useful Data – other than calendar and weather, what actual useful information do you need on this type of board (And don’t tell me quote of the day!)
  • Internet access – your dashboard computer is going to need internet. That means either a wired connection or wifi. Once you’ve seeded this fact, why even run the dashboard software on the Raspberry Pi? Just have it hit a URL on a different, more high powered server.
  • Form Factor – The current setup has way too many cords and cables, and (continuing from the previous point) if you’re just hitting a webpage on a server on the internet, really you’re just building a screen display unit, which can be done sleeker, smaller, and better by manufacturers who do this for a job – like an iPad or other Android tablet. Plus going tablet allows you to use touch which makes interacting with the device easier.

1

Too much cruft!

Software

687474703a2f2f692e696d6775722e636f6d2f31724a754642712e6a7067Setting up the actual software that runs the dashboard was pretty fun – probably the best part of the project for me. I enjoyed configuring the Atlasboard widgets to my liking. I had to edit the Calendar app to work like I wanted it to with our meal planning web service. I also wanted a better weather widget, which I wrote and released on GitHub.

After using it for awhile, I actually took this down out of our kitchen. The benefits were not worth the costs. I enjoyed the project though – especially being able to release an open source widget for Atlasboard – and will maybe revisit the idea in the future.

Mapping NBA Player Movement in 2D and 3D

Having developed methods to get the NBA player shotchart data and player movement data into ArcGIS, I wanted to explore different ways to visualize the player movement data with webmaps, ArcGIS application templates, and 3D web scenes. I also wanted to look at an entire game and find ways to identify where on the court players spend the most time and what lanes or routes players are more likely to travel across the court. For this post, I’m going to describe some approaches to visualize an entire game of movement data with common GIS techniques and tools and use some really cool GIS visualization techniques to display the data and compare player movement maps.

For this post, I looked at the movement data for the entire game between the Oklahoma City Thunder and Detroit Pistons from November 27, 2015. I aligned the data so that the northern end of the basketball court is the Thunder’s offensive end and the southern end of the court is the Piston’s. I filtered the data to remove duplicate timestamps for each player on the court. This has the effect of filtering out moments where game action is occurring but the clock is not moving (for example, foul shots). This is what I wanted to do because I was interested in where players spend time on the court when the game is in progress.

My first thought was to turn player movement points into player movement lines. For every unique event number in unique_event_list, I converted the point features for that event into line features using the PLAYER_ID as the line field. Then, I updated the TEAM_ID, QUARTER, and START_TIME for each line feature to allow me to filter on player name and time. Here is the function I wrote to create the line features.

def create_line_features(in_fc, gdb, unique_event_list, dictionary):
    for event in unique_event_list:
        event_lines = os.path.join(gdb, 'event_'+str(event))
        if arcpy.Exists(event_lines) == False:
            print('Creating lines for event ' + str(event))
            layer = 'in_memory\\event_lyr_' + str(event)
            query = '"EVENT_ID" = ' + str(event)
            arcpy.MakeFeatureLayer_management(in_fc,layer,query,"","")
            arcpy.PointsToLine_management(layer, event_lines, "PLAYER_ID", "GAME_TIME", "NO_CLOSE")
            arcpy.AddField_management(os.path.join(gdb, 'event_'+str(event)),"TEAM_ID","LONG", "", "", "")
            arcpy.AddField_management(os.path.join(gdb, 'event_'+str(event)),"QUARTER","SHORT", "", "", "")
            arcpy.AddField_management(os.path.join(gdb, 'event_'+str(event)), "START_TIME", "TEXT", "", "", 30)
            #Get Start time of lines
            with arcpy.da.SearchCursor(layer, ('GAME_TIME', 'QUARTER')) as cursor:
                first_time = cursor.next()
            #Add start time of lines to line features
            with arcpy.da.UpdateCursor(os.path.join(gdb, 'event_'+str(event)), ('START_TIME', 'PLAYER_ID','TEAM_ID', 'QUARTER')) as cursor:
                for row in cursor:
                    row[0] = first_time[0]
                    row[2] = dictionary[row[1]]
                    row[3] = first_time[1]
                    cursor.updateRow(row)

What does a single event look like as lines?

For single event, I’ve isolated the basketball (orange line), Kevin Durant (blue line), and Ersan Ilyasova (red line). During this event at about 2 minutes into the game, Kevin Durant gets 2-points after driving by Ersan Ilyasova.


View larger map

When I look at this webmap, it makes me think that ArcGIS and webmaps could be really powerful tools for coaches to do X’s and O’s and diagram plays.

What does this look like for an entire game?

I concatenated all the events into a single feature class. The geodatabase I created above (gdb) contains a set of line features for every unique event and I appended every feature class into an empty features class (output_line_fc) and then deleted the individual event feature classes.

def append_lines(gdb, output_line_fc):
    arcpy.env.workspace = gdb
    fc_list = arcpy.ListFeatureClasses('*',"Polyline")
    arcpy.CreateFeatureclass_management(gdb, os.path.basename(output_line_fc), "POLYLINE", fc_list[0],"DISABLED","DISABLED","PROJCS['WGS_1984_Web_Mercator_Auxiliary_Sphere',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Mercator_Auxiliary_Sphere'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',0.0],PARAMETER['Standard_Parallel_1',0.0],PARAMETER['Auxiliary_Sphere_Type',0.0],UNIT['Meter',1.0]];-20037700 -30241100 10000;-100000 10000;-100000 10000;0.001;0.001;0.001;IsHighPrecision","#","0","0","0")
    arcpy.Append_management(fc_list, output_line_fc, 'NO_TEST', '', '')
    for fc in fc_list:
        arcpy.Delete_management(fc)

This is what the entire game looks like.


View larger map

The red lines are the Detroit Pistons, the blue lines are the Oklahoma City Thunder, and the orange lines are the basketball. It’s a real mess! However, what you notice is that at the south end of the basketball court, the red lines cover a wider area and at the north end of the court, the blue lines cover a wider area. The south end of the court is the Piston’s offensive end and the north side is the Thunder’s.

It gets really interesting when you start to isolate the player movement lines for individual players. For example, this is Ersan Ilyasova’s game movement map.


View larger map

Notice that there are some lines that trace around the 3-point arch on the Piston’s offensive end.

Here is Andre Drummond’s movement map.


View larger map

Andre plays center for the Pistons. His movement is confined to the center of the basketball court and on the defensive end, he stays within the free throw lane. If you open the webmap, you should be able to toggle between some of the other players on the Pistons and Thunder and also see the movement lines for the basketball.

How do you compare two players’ maps and what could that comparison tell you?

I really quickly created a player movement map comparison using the ArcGIS swipe template. Here I compare Kevin Durant’s and Marcus Morris’ player movement charts. From looking at different maps, I kind of inferred that Marcus and Kevin were assigned to guard each other.


View Larger App

Still, I didn’t think that this was the most clear comparison, so I converted the player movement data to rasters that represent the amount of time each player spends at a given cell. These rasters I’ll refer to as frequency maps because they correspond to how frequent a player visits a given location on the court. I did this for every player in the game.

def create_point_density_raster(in_points, gdb, player_list, player_dictionary):
    for player in player_list:
        query = '"PLAYER_ID" = ' + str(player)
        ras_name = 'player_movement_rasters_'+player_dictionary[player]
        print('Creating ' + str(ras_name))
        ras_name.replace(' ', '_')
        ras_name = re.sub('[^0-9a-zA-Z]+', '_', ras_name)
        out_ras = os.path.join(gdb,ras_name)
        if arcpy.Exists(out_ras) == True:
            player_lyr = 'in_memory\\'+str(player)+'movement_map'
            print('Making feature layer.')
            arcpy.MakeFeatureLayer_management(in_points, player_lyr, query, '#', '#')
            print('Getting point density raster '+ ras_name)
            arcpy.gp.PointDensity_sa(player_lyr, "NONE", out_ras, 2.5, "Circle 10 MAP", "SQUARE_KILOMETERS")

That same player movement comparison between Kevin Durant and Marcus Morris is a little bit clearer when looking at the data for these two players as frequency maps.


View Larger App

I was going to leave that as the only comparison app, but then I found the Compare Analysis template. With this template, I decided to compare the frequency maps of the Thunder’s starting five. Check it out.


View Larger App

It’s cool to see where each player spends the most time on the court and how that compares to their teammates. I think it’d be really cool if I could quantify the entropy of player movement. Who covers more ground? Who tends to be more stationary on the court? Perhaps a topic for another day.

What do these maps look like in 3D?

I explored (and am still exploring!) visualizing player movement as a 3D map with ArcScene and CityEngine web scenes. I created a 3D scene in ArcScene using the frequency maps. I extruded the frequency maps and put the starting five for both the Thunder and the Pistons into a group layer in ArcScene. I saved the scene document (sxd) and published it as a web scene.

sxd = 'C:/NBA/PlayerMovementScene.sxd'
webscene = 'C:/NBA/PlayerComparisons.3ws'
arcpy.ExportTo3DWebScene_3d(sxd, webscene)

Then I uploaded it to my ArcGIS Online account. Take a look.

Definitely Explore in full viewer for the full effect. In addition to viewing an individual frequency map you can compare player frequency maps.

3D Player Movement Comparison

It’s a pretty fun way to visualize player movement.

Again, I feel like I’m just scratching the surface with what is possible here. The code and data are shared on github if you’re interested. If you have any ideas or questions, leave a comment!

Happy New Year 2016

goals-smallI don’t like new year’s resolutions. They are too often forgotten a few weeks down the road. I do, however, like to have short and long-term goals, and the beginning of a new year is a great time to reflect on what has been accomplished, and what to do in 2016.

2015 Completed List:

  1. 12 Gavinr.com blog posts (met goal of 12, with help from guest blogger Greg Brunner)
  2. Read 21 books (met goal of 15)
  3. Volunteered at least 60 hours (goal of 30)
  4. Contributed 2 small patches to Open Source Dojo JavaScript project (goal of 1)
  5. Released four new WordPress Plugins
  6. Gave three conference presentations – two at Esri Dev Summit and one at Esri User Conference
  7. Built and setup a Raspberry Pi kitchen dashboard (blog post coming soon)
  8. Released some open source projects on GitHub: atlasboard-weather, geojson-viewer, and wab-widget-search

Of course there were goals I had that I didn’t reach, including running and blogging.

Goals for 2016:

  1. Read 20 books
  2. Learn VIM – Be using at least 10 unique VIM commands in SublimeText Vintage mode daily
  3. 12 Gavinr.com blog posts
  4. 60 excercise activities (walking, running, or biking)
  5. Contribute to at least 2 open source projects
  6. Release 2 new WordPress Plugins

The Geography of Basketball, Part II: Watching the Game in ArcGIS

A guest post by Gregory Brunner

I wasn’t planning on writing another blog on this topic so soon, but a few days ago I was looking into how to turn the gameid field in the NBA data into the actual game date, teams involved, etc., and I stumbled upon this:

For about a day, I thought I found something that I wasn’t supposed to find. Then I Googled NBA Movement API and found this amazing post by Savvas Tjortjoglou on How to Track NBA Player Movements in Python. That same day, I found NBA Player Tracking. All this amazing player movement data is out there for consumption. I had to explore it!

All I really wanted to do was get this data into ArcMap, see if I can replay the data using the time slider, and then make some webmaps. I wanted to go from watching this:

(That’s Russell Westbrook hitting a 3 point shot. You can access that data as json here.)

to exploring ways to replay the data in ArcMap:

(That’s the first 90+ seconds of the first quarter of the game in ArcMap played back in 10 seconds!)

and also maybe play around with different ways to render players and moments on the court.


View larger map

That’s Russell Westbrook hitting a 3 point shot.

So how do you go from the NBA player tracking video to working with the data in ArcMap and ArcGIS Online?

I decided to stay with Russell Westbrook and look at the first game he played in the 2014-2015 season. That game was on October 29, 2014 against the Portland Trailblazers and all of the data we’ll look at here is from that game. Note that the game was played in Portland, not Oklahoma City, as you might infer from my court image.

Picking up on Savvas Tjortjoglou’s post, I read in event data from from the stats API by passing a specific gameid and eventid.

event_url = 'http://stats.nba.com/stats/locations_getmoments/?eventid=%s&gameid=%s' % (eventid, gameid)
response = requests.get(event_url)
home = response.json()["home"]
visitor = response.json()["visitor"]
moments = response.json()["moments"]
gamedate = response.json()["gamedate"]

The variables home and visitor are dictionaries containing information on the players involved in the game. Moments are the events on the court for each player. The gamedate is the date of the contest.

I used the home dictionary and the visitor dictionary to create a team dictionary and a player dictionary so that later I could apply those to my feature class in the form of a player domain and a team subtype.

#Create team dicitonary
team_dict = {}
team_dict[home['teamid']] = home['name']
team_dict[visitor['teamid']] = visitor['name']
team_dict[-1] = 'Basketball'
#Create player dictionary
d = {}
d[-1] = 'Basketball'
for h in home['players']:
    d[h['playerid']] = h['firstname'] + ' ' + h['lastname']
for v in visitor['players']:
    d[v['playerid']] = v['firstname'] + ' ' + v['lastname']

Then, for every moment in the data, I parsed the data into a tuple that I used to populate my game event feature class.

    coords = []
    for moment in moments:
        quarter = moment[0]
        for player in moment[5]:
            player.extend((moments.index(moment), moment[2], moment[3]))
            clock_time = create_timestamp(quarter, date, player[6])
            ct = datetime.datetime.strftime(clock_time, '%Y/%m/%d %H:%M:%S.%f')[:-5]
            player_data = (player[0], player[1], player[2], player[3], player[4], player[5], player[6], player[7], ct)
            coord = ([10*(player[3]-25), 10*(player[2]-5.25)])
            coords.append((coord,)+player_data)

Notice that I scaled the coordinates by a factor of 10 and also shifted the x and y positions slightly. I did this to put the movement data in the same coordinates as the shot chart data, which was in units of feet x 10, with the (0,0) point being the center of the hoop. Now that I think about it, I should probably be putting the shot chart data in the coordinate frame of the player movement data as units of feet will make more sense.

Next, I created a feature class.

def create_feature_class(output_gdb, output_feature_class):
    feature_class = os.path.basename(output_feature_class)
    if not arcpy.Exists(output_gdb):
        arcpy.CreateFileGDB_management(os.path.dirname(output_gdb),os.path.basename(output_gdb))
    if not arcpy.Exists(output_feature_class):
        arcpy.CreateFeatureclass_management(output_gdb,feature_class,"POINT","#","DISABLED","DISABLED", "PROJCS['WGS_1984_Web_Mercator_Auxiliary_Sphere',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Mercator_Auxiliary_Sphere'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',0.0],PARAMETER['Standard_Parallel_1',0.0],PARAMETER['Auxiliary_Sphere_Type',0.0],UNIT['Meter',1.0]]","#","0","0","0")
        arcpy.AddField_management(output_feature_class,"TEAM_ID","LONG", "", "", "")
        arcpy.AddField_management(output_feature_class,"PLAYER_ID","LONG", "", "", "")
        arcpy.AddField_management(output_feature_class,"LOC_X","DOUBLE", "", "", "")
        arcpy.AddField_management(output_feature_class,"LOC_Y","DOUBLE", "", "", "")
        arcpy.AddField_management(output_feature_class,"RADIUS","DOUBLE", "", "", "")
        arcpy.AddField_management(output_feature_class,"MOMENT","LONG", "", "", "")
        arcpy.AddField_management(output_feature_class,"GAME_CLOCK","DOUBLE", "", "", "")
        arcpy.AddField_management(output_feature_class,"SHOT_CLOCK","DOUBLE", "", "", "")
        arcpy.AddField_management(output_feature_class,"TIME", "TEXT", "", "", 30)

Then, I pushed the game data into the feature class.

def populate_feature_class(rowValues, output_feature_class):
    c = arcpy.da.InsertCursor(output_feature_class,fields)
    for row in rowValues:
        c.insertRow(row)
    del c

I used the game date for YYYY-MM-DD to create a clock time string. In ArcGIS, for the time to make sense, time needs to move forward. So instead of counting down from 720.0 seconds at the beginning of a quarter to 0.0 seconds at the end, I needed to define a scheme where time moved forward. The time scheme I defined is QQ:MM:SS, where QQ is the quarter, MM is the minutes into the quarter, and SS is the seconds. For example, a time of 01:02:30, indicates that we are 2 minutes and 30 seconds into the first quarter. Here’s how I created the timestamp.

def create_timestamp(quarter, gamedate, seconds):
    m,s = divmod(720-seconds, 60)
    ms = round((s-int(s))*100)
    t = datetime.time(int(quarter), int(m), math.floor(s), ms*10000)
    dt = datetime.datetime.combine(gamedate, t)
    return dt

The NBA data is interesting because there are a lot of coded values. Players and teams have numerical IDs. Above, I created a dictionary of players and teams. I used the players to create a “Players” domain on my feature class.

def create_player_domain(gdb, fc, player_dict):
    domName = "Players"
    inField = "PLAYER_ID"
    arcpy.CreateDomain_management(gdb, domName, "Player Names", "TEXT", "CODED")
    for code in player_dict:
        arcpy.AddCodedValueToDomain_management(gdb, domName, code, player_dict1)

    arcpy.AssignDomainToField_management(fc, inField, domName)

And I used "TEAM_ID" as the subtype on my feature class:

def create_team_subtype(gdb, fc, subtype_dict):
    arcpy.SetSubtypeField_management(os.path.join(gdb,fc), "TEAM_ID")
    for code in subtype_dict:
        arcpy.AddSubtype_management(os.path.join(gdb,fc), code, subtype_dict1)

It's pretty neat how we can apply these GIS data management fundamentals to the NBA data!

I also added a line to remove duplicate moments from the feature class because when I started concatenating multiple events, I noticed that there were some duplicates.

arcpy.DeleteIdentical_management("Game_0021400015_Event_1_10", "Shape;TEAM_ID;PLAYER_ID;LOC_X;LOC_Y;RADIUS;MOMENT;GAME_CLOCK;SHOT_CLOCK;TIME", "", "0")

So what does this data look like in ArcGIS?

At about 1 minute and 6 seconds into the game, Wesley Matthews took a 3 point shot (You can watch the footage here at on.nba.com. The shot is the first video in the sequence).

Wesley Matthews Footage

What does this look like in ArcMap?

Wesley Matthews Shooting

This is a roughly 1 second interval of data right after Wesley Matthews released the shot.

He missed.

He missed and the players moved in for the rebound. I used this project to make the movie above. I have also shared this map document (MXD) as a map package (MPK) on my github site. Take a look at the map package in ArcGIS if you're interested.

What does a single event look like as a webmap?


View larger map

This is the webmap for Event 346 (Russell Westbrook hitting a 3). Click on a feature to see the attribution. There are over 7,000 features in this single event and this event spanned less than 10 seconds on the game clock!

In my webmap for Event 346, I can isolate Russell Westbrook and the ball using a filter in ArcGIS Online. It looks like this.


View larger map

That's where Russell Westbrook hits a 3. I've applied the heat map effect to the basketball.

I got ambitious when I wanted to make a video and time-enabled webmap, so I concatenated Events 1 - 10 from the same game. Those are the events I used for the video at the beginning. I published the data as a time-enabled webmap. The time-slider won't appear till you View larger map.


View larger map

The map looks like a mess until the slider appears. There are over 30,000 features in this feature service. I'm sharing it as a hosted feature service, which I did not realize until Gavin noticed, but the hosted feature service stripped out the sub-seconds on the TIME field. I don't know why this happened because for the Event 346 webmap, I can see the sub-seconds. This impacts playing back the game moments as does the fact that the default time-slider does not display data in less than 1 second intervals. We would have to create a custom slider or playback capability to accommodate playing back data of this frequency, which we're thinking about doing. We'll write about it after we do it. Still, I'd encourage you to take a look and play around with the time-slider and data yourself. You can identify, query, and render the data in many different ways!

All that being said, this has all been really fun for me! I can't believe all this data is out there. We're really just scratching the surface. If anyone would like to see the code, you can find it at my github page. I have also shared a map package there if you want to see the data in ArcGIS. Definitely let me know if you have any problems, questions, or insights! In addition to exploring how to improve the playback capability, I'm thinking about doing a post that explores some ArcGIS spatial analysis or space-time analysis and visualization on the data. Let me know if you would be interested in reading about that.

This Week in Esri GitHub

esri-arcgis-js-api-githubEsri has opened up a bunch of repositories on GitHub this past week. To summarize:

JS API now available on GitHub and Bower:

Calcite:

ArcREST, a set of python tools to assist working with ArcGIS REST API for AGS/AGOL/WebMap-JSON, has seen a lot of activity: github.com/Esri/ArcREST

Going to DevSummit in March? Keep an eye on this repo: github.com/Esri/devsummit-feedback

There’s probably other stuff I missed too. Let me know in the comments!

Older Posts

Gavin Rehkemper

JavaScript, WordPress, and GeoDev