Exploring San Francisco's data on crime

San Francisco is among the cities which made their data publicly available to everyone. This is very progressive (but also missing in many other areas in the world) and it shows that a high level of transparency breeds more trust in the work of the public institutions, which are serving all citizens. This allows anyone to feel that they are participating in the system rather than being excluded from it. If an area of the city governance needs improvement, the information is there to allow finding the people with the right expertise who will be willing to be held accountable for their actions, being publicly visible to everyone. This is important, because it has been repeatedly shown that when the institutions in one country are strong and work well together, the country as a whole progresses faster and its people can more easily unfold their full creative potential. If that's not the case, they waste energy on fighting the bureaucracy around them.

It would be nice to see more cities following this positive example. Merely sharing the information is not enough; everyone must feel that their efforts are actively changing it and that there are constantly ups and downs, depending on how we do collectively.

Here we will try to explore one of the datasets on crime and see whether we can find something interesting from it. You can find the dataset here. One would expect that there wouldn't be crime in San Francisco if everything is shared so openly, but this is not the case. It requires a certain degree of faith to admit any form of vulnerability, knowing that it could scare tourists away or keep multinationals unwilling to invest in the city. This could be one reason why other cities aren't accepting this as a good example. But doing so and silently keeping the major events in secret is in disservice to everyone. One day it may become clear that our cities are gradually losing their infrastructure they once had, gradually losing their production facilities, neglecting education and healthcare, the people in need, their identity. If we continue to pretend behind a facade, this would be inevitable.

How many records are in this dataset?

132083 rows x 13 columns

In which period was the data collected?

Between 01.01.2016 and 24.11.2016 (likely still updated).

Can this dataset be held in memory at once?

It takes ≈71.8MB memory to do so.

Which was the most common incident category in San Francisco this year?

Larceny/theft35251
Other offenses17196
Non-criminal15760
Assault12061
Vandalism7485
Vehicle theft5680
Warrants5203
Burglary5080
Suspicious occ5027
Drug/narcotic3846
Missing person3746
Robbery2924
Fraud2275
Secondary codes1624
Trespass1589

Which was the most common incident description?

Grand theft from locked auto15398
Aided case, mental disturbed4077
Lost property3972
Petty theft of property3815
Battery3778
Malicious mischief, vandalism3656
Petty theft from locked auto3532
Stolen automobile3202
Drivers license, suspended or revoked3127
Found property2765

In which districts crime was most common?

Southern24981
Northern17588
Mission16984
Central15414
Bayview12615
Ingleside10153
Taraval9858
Tenderloin8807
Richmond7867
Park7815

At what time of the day happened the most incidents in the Southern district?

17:00-18:001765
18:00-19:001689
16:00-17:001554
19:00-20:001504
15:00-16:001419
00:00-01:001406
11:00-12:001378
14:00-15:001293
20:00-21:001266
21:00-22:001261
13:00-14:001256
12:00-13:001182
22:00-23:001129
10:00-11:001020
09:00-10:00902

Can we see the distribution by the hour?

Crimes by hour for San Francisco, 2016

At which addresses were incidents happening most frequently?

800 block of Bryant St3049
800 block of Market St1219
1000 block of Potrero Av572
900 block of Market St476
0 block of Unitednations Pz420
500 block of Johnfkennedy Dr418
600 block of Valencia St355
3200 block of 20th Av353
1100 block of Fillmore St346
300 block of Eddy St323
100 block of Ofarrell St312
16th St / Mission St311
0 block of 6th St308
800 block of Mission St297
700 block of Mission St296

Where is Bryant Street?

Latitude: (-122.40455785073601, -122.402771389219) Longitude: (37.774431418760294, 37.775859961640798)

On which days were the most incidents?

01/01/2016529
10/08/2016520
04/01/2016516
01/29/2016503
06/25/2016503
02/04/2016489
07/01/2016488
02/01/2016486
05/23/2016482
01/23/2016481

How were most cases resolved?

None94083
Arrest, booked35071
Unfounded1416
Juvenile booked939
Exceptional clearance327
Arrest, cited141
Cleared-contact juvenile for more info54
Not prosecuted20
Psychopathic case14
Located11
Juvenile diverted2
Juvenile cited2
Complainant refuses to prosecute2
Prosecuted by outside agency1

Among the unresolved cases, on which day of the week were registered most of them?

Friday14860
Saturday14104
Monday13127
Tuesday13037
Thursday13029
Wednesday12981
Sunday12945

Can we see a map of all cases?

Map of crime in San Francisco, 2016

As you can see, open datasets can help to address many interesting questions.