Monday, August 29, 2016

Develop and Implement a Backup Strategy

I originally wrote this post for a software company’s blog in 2014. This company was bought out and recently their blog and website have been removed from the web permanently. I am reposting it here for posterity. 

I’ve been working with computers both professionally and personally since the late 1980’s. Quite a lot of the technology has changed. My first computer with a hard drive had a whopping 20 megabyte drive. The laptop computer I am typing this on right now has a 320 gigabyte hard drive. That’s 16,000 times more storage space. All that over the course of about 25 years.

One thing that hasn’t changed is the need to back up your data. While computers and their storage medium are much more reliable than they were back when I started, they still are far from infallible. In my day I’ve seen drives fail, data files become corrupted, network drives overwritten, drives accidentally formatted, files mistakenly deleted and a virus encrypt and lock out user’s data.

Let me ask you a question. Just how far back would you be set back  if the files on your workstation disappeared? How would you do your job? How much would this affect your agency? If you’re like me, a loss of these files would create quite a problem. This is why it’s important to develop a sound backup strategy.

In this post I am going to look at a personal backup strategy. That is, a strategy to backup files on your workstation. I’m not going to cover server backups, shared database backups, etc. I know for some of you, you have a competent IT department that handles backups on the network. However, you might find that while they back up your shared network folders they often don’t backup anything from a user’s machine. Even if they did, don’t be surprised if getting them to recover  from a backup tape that one Word document you accidentally deleted is not very easy or timely.

This is why I firmly believe that you should be in charge of your own backups. Let’s cover a bit of terminology first. There are different types of backups. They are:

Full or System Image Backup - This backup backs up your entire drive; your operating system, applications, settings and user files. You could take a brand new empty hard drive restore your entire machine to just like it is now.

User Files or Selective Backup - This backup only backs up the user’s files and not programs, system settings or operating system. It would backup the files in your My Documents folder, Desktop files, etc.

Complete Backup - Backs up all the files selected (either Full or User Files) every time the backup is run.

Differential Backup - Only backs up the files that have changed since the last backup. The first backup would have all the selected files and subsequent backups would have only the files that have changed. This saves time when running the backup compared to the Complete Backup.

There are a number of ways to backup. You can backup to CD or DVD media, to a tape backup device, to a network, to another hard drive or to USB storage. There are advantages and disadvantages to each type of backup media. There’s a pretty good Wikipedia article that outlines the advantages of each of them here.

However, there are a number of things I do want to cover when it comes to developing your own backup strategy. One, you need to develop a method of backups that is easy to do and to do it regularly. Two, this strategy needs to include multiple, redundant backups. You need to create several backups using different medium and to store those backups in different places. Lastly, you need to execute your backups like clockwork.

Here’s a peek at my workstation backup strategy.

I currently only do a Selective Backup of my user files. Until I can get them to buy me my own personal Network Attached Storage device or RAID array I don’t have the space to do a complete backup. While this isn’t ideal, it has the advantage that a drive failure would mean a fresh clean operating system and application install. Choose your poison.

Every single workday, I use a large capacity USB drive to copy all of the folders and files in my My Documents folder. I use a batch file with the XCOPY command to copy every directory and every file in it to the USB drive. I also use the /d switch to copy only those files that are newer than the ones on the USB drive. This option cuts down on the time required to complete the operation from over two hours to just under 30 minutes once you’ve made the initial backup. This is for just under 16GB of user data. Plug the drive in, double click the batch file and wait for it to complete. Couldn’t be simpler. This handles all the mission critical stuff that I would need to get up and running if need be. Plus, since the files are just copied to the USB drive, it’s super easy to find a file you need.

Every week, I then burn all the files from the My Documents directory as well as all the files on a shared network folder to DVD /R’s. These disks are then stored in a fireproof safe. I also periodically make additionally copies of these disks and store them off site. I use a repeating To Do item on my Task list to remind me to accomplish the weekly backups. I also archive old project files and data yearly to DVD /R and store them in a fireproof safe by year.

More than once I’ve had to go to my backups to find something that inadvertently got deleted or corrupted. The time spent backing up is nothing compared to the time and headache saved by being able to retrieve a file that went AWOL.

Do you have a backup strategy? Do you regularly backup using it?

Monday, August 22, 2016

How To Get Things Done: Thoughts on Task Lists for Crime Analysts

I originally wrote this post for a software company’s blog in 2014. This company was bought out and recently their blog and website have been removed from the web permanently. I am reposting it here for posterity. 

Charts for the Chief, requests from Patrol supervisors, inquiries from neighboring agencies and information requests from the public, all of these and more are seen by crime analysts in the course of their work week. In fact, with all the things that come at a crime analyst it becomes important that you develop a systematic approach to managing your tasks.

When I first became a crime analyst I quickly recognized that I needed to adopt a system of task management. I needed to track incoming tasks, keep up with due dates, prioritize those tasks and ensure that nothing slipped through the cracks.

About this time I heard of a book by David Allen called Getting Things Done. Allen’s book outlines a system for managing tasks in order to be and stay productive. Allen’s book and system became so ubiquitous especially in the tech world that it became known simply as GTD and attained near cult like status. I got a copy of the book and ended up reading it twice and underlining many of it’s passages. In a nutshell, GTD consists of Five Steps.

  1. Capture
  2. Clarify
  3. Organize
  4. Reflect
  5. Engage

Let’s look at what the Five Steps entail one by one.

Capture - In this step, you designate a place for an Inbox, either physical or electronic. All your incoming tasks, projects, etc. are first gathered here.

Clarify - Now, you’ll review the items in your Inbox one by one. Is it something you need to do? If not, trash it or file it. Next, if you can do and item in two minutes or less, then do it right then. Else, you need to determine if you can delegate it or to put it on your list.

Organize - Sort your tasks to appropriate lists. Calls, emails, projects, etc.

Reflect - Set a regular time to go back over lists and determine what to do next, update items with changing priorities, etc.

Engage - Do what’s on your lists.

There’s more to GTD that I can cover here so I would highly encourage you to get and read the book for yourself. That said, let me give you a peek into how I implement GTD for my crime analysis workflow. But a couple of notes before I do. One, while GTD is very popular with tech oriented folks, you can implement a system of GTD in a variety of mediums, using a computer, a notebook or as it was originally incarnated, with a series of folders. Don’t think that you have to use a specific method, just use what you are most comfortable with.

Also, don’t think that you have to do GTD exactly like David Allen does or anyone else for that matter. For instance, I don’t break my task down into lists for phone calls, lists for emails, etc. I also don’t separate projects (tasks with multiple sub-tasks) into a list separately from tasks that consist of a single step. I just don’t need that level of granularity.

Let’s peek at how I implement GTD.

I am very comfortable on the computer. In fact, I am the kind of person who would rather get and receive emails as opposed to phone calls. For this reason, I have implemented my GTD system into Microsoft Outlook. In addition to the main Inbox folder for incoming email, I have added additional folders for Working, File and Wait. An email comes into the Inbox and several times a day I then Clarify those emails by moving them into these folders.

An item that gets moved (Organized) into the Working folder also gets assigned a priority and if necessary a Due Date. If you right click on an email item you can set these easily. Then you will also see these items in your Outlook Tasks (you may need to change your Outlook options to see these in your Outlook Tasks). I also sort the Task list by Due Date so I can see where I stand at a glance.

Items moved to the Filed folder will be then filed appropriately periodically. Items that get moved into the Wait folder are for those items that will come in the future. For instance an email about an upcoming meeting or a task that I might be waiting on someone else to complete.

For those items that don’t come in via email, I create an Outlook Task. I use Due Date, Priority and the notes field heavily.

I’ve also found it somewhat helpful to keep some separate hanging files with the same labels on them. The reason being is that at least one person will generate a piece of paper associated with a Task such as a form to complete. I will create an Outlook Task for that item and then stick the piece of paper into the corresponding hanging file for completion when I complete the Task.

I have also found it convenient to send an email to myself when I am away from my desk when I need to add something to my Task list. For instance, if I am in a meeting and someone assigns me a task to complete I can use my smartphone or iPad to send an email to my office email account. That way, when I get back at my desk the item is in my Inbox (Captured) for later Clarification, Organization and ultimately Engagement.

GTD is just one way to organize your tasks. Whether you use GTD or something else, the most important thing is to have a system to handle the myriad of tasks that will come your way. Find a system, refine it and stick with it.

How do you manage your workflow?

Monday, August 15, 2016

Simple Crime Series Predictions

I originally wrote this post for a software company’s blog in 2014. This company was bought out and recently their blog and website have been removed from the web permanently. I am reposting it here for posterity. 

One thing that often works in our favor in our criminal investigations is that the targets of those investigations are people, and quite often people are creatures of habit. We all have our routines; we get up at certain times, go to work at certain times and come home at certain times.

Criminals also have their routines. Given that they are sometimes dealing with character flaws that make them unsuited for normal jobs such as unreliability, substance abuse and lack of stable environments their routines may not be quite as set as ours. However, even they have routines.

These routines are often cyclical. Additionally, once a criminal discovers a modus operandi (M.O.) that is successful they will often stick with that M.O. until something interrupts that cycle.

In this's post we're going to look at a simple method for predicting when a criminal will be likely to strike again in a crime series. First, we should answer what is a "crime series"?

A crime series is a group of crimes that are committed by the same offender or group of offenders working together. The armed robber who hits a number of convenience stores over a period of weeks is a crime series. A crime series is not all similar crimes for instance, all convenience store robberies, but only those that can be attributed to the same offender.

This is one reason that it is extremely important that your officers collect very detailed information about the exact M.O. used in a crime. Criminals don't usually identify themselves each time they commit a crime in a series so we often have to use M.O., physical description, etc. to figure out which ones are likely committed by the same offender.

Often times, criminals who commit a series of crimes fall into the 80/20 rule. This anecdotal "rule" says that 80% of your crimes may be committed by only 20% of your offenders. A criminal committing a series of crimes can often have a measurable impact on your crime rate. Making it a priority to put him out of commission can help make your crime numbers look much better at the end of the year.

Giving your officers some guidance about when a serial offender is likely to strike will improve the chances that they will be successful in stopping him. One way to do this is to make a prognostication about when the offender is more likely to strike next.

There are a number of ways to do this. Since this series of posts is geared towards analysts who may not have much formal crime analysis training, we're going to look at one of the easiest: calculating Mean Days Between Hits.

First lets arrange all the crimes in order of the date of occurrence. These examples are from a real armed robbery crime series I worked on a few years ago. Then we're going to calculate the number of days between occurrence. In the example I've arranged these "hits" into a table that looks like this:

Once we've done this, we're going to calculate the Mean (Average) of the Days Between Hits for the events in the series. I've done this with a spreadsheet using the  Excel AVERAGE function.

The next part, is we're going to calculate the Standard Deviation for the events in the series. In a nutshell Standard Deviation is a measure of the variation of the data points from the average. In statistics, One Standard Deviation encompasses 68% of the data points. This is a pretty good accuracy for our purposes. In our example, the formula used for Standard Deviation is STDEV.

Below our "hits", I've added cells for the Mean, Standard Deviation and then an Earliest and Latest cell.

To calculate the Earliest prediction subtract the Standard Deviation from the Mean. The Latest prediction is calculated by adding the Standard Deviation to the Mean. What these Earliest and Latest calculations mean is that there is a 68% chance that the next event in the series will occur between 12/14/09 and 12/17/09.

Let's look at how our prediction fared when we add the next event in the series.

Our next event was three days after the last event. Using our Latest calculation, this was within the range we calculated. Now, we're going to recalculate our Mean and Standard Deviation using this next event.

This Earliest and Latest calculation isn't significantly different from the first. Using these numbers we can expect our crook to hit from 12/17/09 to 12/20/09. So let's see how we fare when we add the last event in the series.

12/23/09? Six days between hits? What happened here?

What happened is that something interrupted the cycle and caused the next event to take place outside one standard deviation (68%) of the mean. It could be that the crook was sick, jailed, or his car broke down. Even crooks have events outside their control that interrupt their routines.

This demonstrates a weakness in making predictions in crime series. However it also demonstrates an important point when making these prognostications. That is, when making these predictions it is important to remember to allow for something out of the ordinary. Language such as "If the series continues as it has we can expect him to hit on this date..." can help to explain these anomalies.

A couple of caveats: There are other more sophisticated ways to calculate these predictions that may be more accurate. However, Mean Days Between Hits is a good way to get in the ballpark and doesn't take an understanding of advanced statistics to calculate.

Also, the more events you have in a series the more accurate your calculations are likely to be. You should probably have 4 or 5 events in a series before you try to predict when the next event will be.

What types of crime series do you think lend themselves to this type of analysis?

Monday, August 8, 2016

Let the Problem Oriented Policing Center’s POP Guides guide you

I originally wrote this post for a software company’s blog in 2014. This company was bought out and recently their blog and website have been removed from the web permanently. I am reposting it here for posterity. 

There are a lot of tasks that fall to crime analysts in a police agency: compiling statistics, generating BOLO’s, producing reports. But there is one task that crime analysts are particularly suited for but is often overlooked; helping to mitigate troublesome crime problems. Patrol officers are often too busy answering calls, detectives have individual cases to work, and agency heads are often ensuring that their agency has adequate resources. Often, only the crime analyst can focus on individual problems for a long enough period of time to develop an adequate solution.

Let’s look at what Problem Oriented Policing is. Professor Herman Goldstein states:

“Problem-oriented policing is an approach to policing in which discrete pieces of police business (each consisting of a cluster of similar incidents, whether crime or acts of disorder, that the police are expected to handle) are subject to microscopic examination (drawing on the especially honed skills of crime analysts and the accumulated experience of operating field personnel) in hopes that what is freshly learned about each problem will lead to discovering a new and more effective strategy for dealing with it.”

In Problem Oriented Policing, a “crime problem” is usually pervasive, repetitive and resists easy solutions. You’re not going to go out with a typical police response and make the problem go away. It’s going to take a carefully crafted approach and some outside the box thinking to make the problem go away or at least be reduced to a less problematic level.

One thing I wanted to accomplish with my blog posts this year was to give some guidance to crime analysts from smaller agencies that might not have been able to get the kind of formal crime analysis training that larger agencies provide. Looking at Goldstein’s definition of problem-oriented policing above, some analysts might feel they don’t have the formal schooling to conduct “microscopic examinations” of crime problems because they don’t feel their analytical skills to be very finely “honed”.

Don’t worry about this. The Problem Oriented Policing Center has done a lot of the heavy lifting by providing some excellent resources that you can use to guide your examination of crime problems and help you with the collective wisdom of a lot of analysts way smarter than me. One of these resources is the excellent series of POP Guides.

POP Guides come in three types, Problem Specific Guides, Response Guides and Problem Solving Tool Guides. The last two deal with responses to problems and analytical techniques or tools that are used to deal with problems in general. The first type, Problem Specific Guides is what I find most useful. There are over 70 Problem Specific Guides that cover a wide variety of crime problems that police agencies face from the common such as False Burglar Alarms (POP Guide #5) to the more obscure Student Party Riots (POP Guide #39).

Each one of these Problem Specific Guides is organized into sections that define the problem and help you to know the questions to ask to understand your local problem, look at the advantages and disadvantages of potential responses, a bibliography of resources and academic research into these problems and handy summary table of potential responses to specific problems.

As you examine your local crime problem with a POP Guide to assist you, you can leverage the wisdom of other agencies, other analysts and criminologists in analyzing your local crime problem. You can cut straight to responses that are more likely to be effective by relying on these other agencies’ experiences with similar problems.

In addition to being excellent and easy to use resources, POP Guides have another advantage especially for small, often times cash strapped agencies. These guides are free. You can view them on the web, download a PDF of ebook copy or even order printed copies for free.

Look over the list of Problem Specific Guides. What guides would be useful in mitigating problem crimes in your jurisdiction?

Monday, August 1, 2016

Simple Temporal Analysis Heat Maps in Excel

I originally wrote this post for a software company’s blog in 2014. This company was bought out and recently their blog and website have been removed from the web permanently. I am reposting it here for posterity.

I recently had a conversation with someone who was doing research into how crime analysts work. One question he had concerned the acceptance of analytical products by officers. I told him that it had been my experience that officers were hungry for direction in the form of analytical products. They pretty readily accept analysis that would help them improve their effectiveness in their crime fighting mission.

One of the easier analysis you can conduct is a basic temporal analysis. For example, do your officers know what days of the week certain crimes are more likely to occur? At the agency where I work, Thursdays are consistently one of the busiest days for Calls for Service. Because this trend has been so consistent over the years, we scheduled the officers rotating days off to where no officers were regularly off on Thursdays.

But day of week isn’t the only temporal factor, time of day is as well. Certain types of crimes are more likely to occur at certain times of the day. Vehicle burglars are more likely to operate during the overnight hours when cars are more likely to be parked for longer periods and people are more likely to be asleep and not paying attention to their vehicles. Residential burglars most often operate during the day when people are more likely to be at work or school and not at home. But, this temporal routine goes out the window on the weekends when people are more likely to be home regardless of the time of day.

Crime is often cyclical because people, both victims and offenders have adapted to temporal routines. Because some of these temporal cycles are so regular, we can use temporal analysis to discover patterns in these cycles that allow us to better ensure that we can bring the appropriate resources to bear to deal with them.

In the example in today’s post, we’re going to look at some of the temporal patterns in Driving While Intoxicated arrests. Imagine in this scenario you are a police patrol supervisor and you want to ensure that your officers who conduct DWI enforcement are working during the times and days that they are most likely to arrest a DWI offender. One way to do this is with a simple temporal analysis called a “heat map”. This chart will quickly tell us the times and days when most DWI arrests are made.

I’m pretty fortunate to have access to some pretty sophisticated crime analysis software. But even if you don’t have specialized crime analysis software, it’s pretty easy to create a temporal heat map using nothing more than Microsoft Excel.

What I did to prepare my data is I organized the times of day and day of week of DWI cases into an Excel spreadsheet. One way is to manually input this into a spreadsheet but another way would be to export a tabular listing of this data from your records management system. What we want is a spreadsheet that has the time of day for each DWI arrest in one column and the time of day in another. It should look something like this:

Now that we have our data set, we can populate the table for a temporal heat. We’re going to use a Pivot Table and Conditional Formatting to generate this heat map. Select Insert Pivot Table from the ribbon menu in Windows or go to Data > Analysis > Pivot Table on a Mac. Then you need to set your Row Labels to the Hour field, the Column Labels to the Day and Values to Count of Day. Now you should get a pivot table with the numbers in each cell like this:

Now we’re going to use Excel’s Conditional Formatting to give each cell a color that corresponds to the value in each cell. Select the cells within the pivot table then go to the Home Ribbon and select Conditional Formatting > Color Scales and then choose an appropriate color scale. I prefer the one that uses hotter colors (orange, red) for bigger numbers and cooler colors (green) for smaller ones.

Once you’ve done this you should get a temporal heat map that looks like this:

This heat map makes it pretty easy to see that most DWI arrests are occurring on Fridays, Saturdays and Sundays between 1AM and 3AM. This is kind of information that a patrol supervisor could use to make actionable decisions about scheduling officers to work DWI enforcement. A couple of cautions are in order. If possible, try to avoid looking at just time or just day by themselves. Just because one day is hotter than another or one time is more prevalent does not mean that that particular time on that particular day is most prevalent. Combining both time and day into a heat map gets around this problem.

Also, if you are basing your analysis on known cases or arrests as we did in this one, it doesn’t mean that more offenses aren’t happening that you aren’t discovering. If we aren’t making many DWI arrests during the daytime is it because there are much fewer DWI’s out there or are we just not looking for them at that time or both?

What kinds of crimes could you understand better with a temporal heat map?