Bob's Adventures in Wireless and Video Headline Animator

Thursday, April 28, 2016

The Hidden Costs of Body Worn Cameras…

Citizens, politicians, civil rights advocates, prosecutors, and police officers want body worn cameras (BWC's). They are an effective tool for law enforcement, providing reliable evidence and deterring bad behavior by both the public and the police.  However, deploying cameras and implementing an effective policy can be difficult both politically and technologically.


A little background

The current generation of body worn cameras function basically like any video camera:  someone has to push the button to turn the camera on and off.  That someone is the police officer who wears the camera.  The video is stored on an SD Card or internal flash, and then downloaded by the officer at the end of his/her shift to a storage repository.  We refer to this as the “dockable” camera model.
In a later article we will address the benefits of next-generation wireless (LTE/Wi-Fi) cameras which are always connected and how they change everything, but for now we will focus on dockable cameras.

The difficulty with body worn cameras is striking the balance between privacy rights, cost of operations, and the collection of meaningful evidence.  Legislators, police departments, police officer unions, and other interest groups all try to draft policy that strike this balance. Generally the result is a long list of rules dictating when the officer needs to turn the camera on and off.
In most cases the resulting policy is a pragmatic balance that allows police to record most interactions with the public where evidence needs to be collected, and where the camera is off when privacy is a concern or when nothing relevant is occurring. 

A typical policy would have the officer activate the camera when they are dispatched for an incident, when they believe a crime or infraction may be in progress, or when they believe a life or property threatening incident is occurring.  It would also have the camera turned off when officers enter a private property without a warrant or without the prior permission of the resident or property owner; when a sexual crime has occurred and the victim requests privacy; when a minor is being interviewed in the absence of parental consent; and similar situations. The officer would also have discretion to turn the camera off for his/her personal privacy.  Effective policy implementation relies on officer discretion.

Then there is the issue of retention, which generally keeps all recorded video for a minimum 30-60 days, with incident-relevant video kept for longer periods, from 1 year to forever.

To oversimplify the positions of various interest groups1 :

  • Privacy advocates want the least amount of recording possible, only recording evidence of actual crimes where only the perpetrator and individuals directly involved are in the field of view. They want the video retained for the minimum time possible, with complete redaction before being released.
  • Social Justice groups want only recording of police actions, exclusive of the public, unless there is evidence which would exonerate someone.  They want evidence of police wrongdoing kept forever, and the rest immediately deleted.
  • Police officer unions want to record only criminal evidence, leaving out anything which might be taken out of context and result in disciplinary actions. Once the video is recorded, police officers  want the right to edit it, then determine how long to keep whatever they think is relevant.
  • Prosecutors would like everything recorded, giving them the ability to select what evidence becomes part of the record.  They would like to keep everything, but only release what they decide is relevant.
  • Courts prefer continuous recording of everything. And everything should be kept forever.
  • News media wants recording of all sensational events, with no redaction. They would keep the video themselves forever once it is released.
  • IT departments and city budget managers would like to record a fixed, predictable, and minimal amount of video, in order to reduce infrastructure costs. They actually drive the pragmatism of the policy.
  • Service providers want to record everything possible and store it forever.

The results of these divergent interests on policy vary dramatically, from a few minutes of video per shift on one extreme to hours of video on the other.


Mathematical Limitations

Let’s assume that all of the interest groups compromise and arrive at a policy right in the middle with the result:

  • 2 hours of video per officer per 10 hour shift is recorded.
  • Cameras record at the relatively low resolution of 720p (1280x720 pixels) at 30 frames per second.
  • The resulting video files are about 2.472 Gigabytes per hour, or 4.944 Gigabytes per shift.
  • All video is saved for 30 days.
  • 6% of all video is deemed to be potential evidence and is retained for 1 year.
  • 2% of all video becomes part of a case file and must be retained for 5 years.

Then let’s use an example of a mid-size agency with 60 officers who work 3 shifts per day with 20 officers per shift, each officer with his/her own camera. (Yes, the officers get no time off in my example):

  • 20 officers per shift X 2 hours of video per shift X 2.472GB per hour of video X 3 shifts per day = 297GB of video per day. This is how much needs to be stored and/or transferred to the cloud each day.
  • 30 days per month of all video = 8.7 Terabytes of video per month (rolling)
  • 6% of video per month is potential evidence = .06 X 8.7 TB = .522TB per month
  • 1 year Long term storage of potential evidence = .522TB x 12 months = 6.264TB per year (rolling)
  • 2% of video per month becomes evidence = .02 X 8.7TB = .174TB per month
  • 5 year Long term storage of evidence files = .174TB x 60 months = 10TB per five years (rolling)

Once you have recorded this video, you have to store it, protect it as evidence following chain of custody rules, make it searchable, and provide it to various constituencies in either raw or redacted from to protect privacy.

Regardless of how you store your video, you will need to purchase the cameras, which cost from $400 to $1,000, depending on the supplier. Then you will need a docking station, which can cost an additional $50 to $200 per port depending on the supplier.  These costs are fixed for both local storage or cloud storage architectures.

This is where a decision has to be made:  do you buy local storage, or do you use a vendor solution in the Cloud.


Local Storage

The math to determine the cost of local storage is pretty simple.  You need to buy a server, disks and tape, provide power, keep it maintained, and provide support.  It needs to be secured, following CJIS2 standards (for both physical and network security).   

While these may seem onerous at first, they are not unlike traditional evidence chain of custody management. Not transiting public networks or using a hosted facility means greatly simplifying the implementation of a complaint facility.

Moving the video from the camera to a dock to a storage system on site is straight forward. Everything can be on the same LAN and is local, so you have plenty of bandwidth.

This could be covered with a single PC Server with 12TB of RAID disk storage, an LTO (Long Term Offline) tape drive, 4-2TB tapes, and software supporting transfer, storage, search and redaction management.  Let’s say this whole system costs $40,000 in capital and about $5000 a year in recurring software and maintenance. Over 5 years the total cost of ownership is about $65,000. Allocating this to each camera it works out to be about $1083 per camera, or about $18 a month.

Of course you need an IT support engineer to maintain the system.  But this will not be a full time job and in most cases can be provided through existing resources.

You need to add in the cost of power and air conditioning as well, which could run about $100 a month.
No other infrastructure is required.



Cloud means using shared servers hosted at a remote facility.  This can reduce the labor and support requirements from IT, although not completely eliminate it, as the cloud service provider maintains the servers, storage, and support infrastructure at their facility.  For small agencies this may seem to be a very attractive option.

Your end to end network, reaching from your facility to the cloud hosting facility, must be CJIS compliant.  At present there are very few such facilities.  

Pricing for cloud storage solutions are advertised from about $79 per month per camera with unlimited storage.  But there are typically additional service fees which can add up to well over $100 per month per camera.  This adds up to $6,000 per camera over 5 years.  Compare this to buying your own storage solution at $1,083 per camera – what would be more affordable to you?

In some cases vendors set a maximum storage allocation per month which is covered by their base service cost.  Once this allocation is exceeded, additional costs (from 6.5 to 12 cents per gigabyte per month) are added.  A number of agencies have been surprised by these charges.  In some cases these charges are waived in the first year and then become payable after the agency has built a large repository of evidence.


How Secure Is your Data?

The CJIS guidelines provide great guidance on implementing security for video evidence which transits the public Internet, exists in shared storage environments and leaves the custody of the agency which created the video.  There is no doubt that these guidelines are well thought through.

However, as we all know, hackers seem to find ways into just about every facility, and CJIS data centers represent an attractive honeypot of data for them.  With all government agencies sharing one massive data center, it is incredibly attractive.  Once one server is compromised, the entire facility is at risk.

Further, the legal admissibility of cloud stored video evidence is still an open topic.  Often, the physical location where the data resides may not be known.  It may have been moved from one jurisdiction to another even though it never left the service provider's network.  Challenges to data access, data tampering, etc. have not yet been vetted through the courts. 

Keeping data local eliminates many of these issues, provided due care has been taken to physically and logically protect evidence files.  A well-engineered local storage system which follows CJIS guidelines and is kept off the Internet (air gapped) may avoid much of the current risk of cloud storage.

This is not to say that cloud storage is any more or less secure than local storage, but there are many unresolved issues with cloud that will only come to closure over time.  It may be better to wait on cloud.


Broadband Bottleneck

A basic assumption to use cloud services is that adequate broadband connectivity exists between the police facility and the cloud facility.  This is where police agencies can get bitten, hard.

Let’s go back to our storage requirement of 297GB of video per day.  A transfer of this video evenly throughout the day equals 12.375GB of video per hour.  To do this you will need a dedicated upload speed to the Cloud service provider of 33Mbps3.  That is UPLOAD speed.  It needs to be DEDICATED, not shared.  And that is persistent, running 24 hours a day, 7 days a week, 365 days a year.

Most broadband connectivity is asymmetric with the download speed being the focus. Upload speeds are typically much lower.  Higher upload speed requires the purchase of high cost business services such as 1Gbps Ethernet.

Based on the most recent FCC Measuring Broadband America4 report for 2015, the mean upload speed for consumer services is about 5Mbps:

Click to enlarge
So generally, you will need to have an Ethernet Virtual Private Line (EVPL) service or a dedicated Metro Ethernet connection between the police facility where the download docks are located and the cloud storage provider location. This type of service can cost thousands of dollars a month and can take months to years to install depending on your location.  This cost has to be added into the calculation of your cloud solution.

The other consideration is that you need to be transferring video continuously.  If there is an interruption in service, you will need to buffer your transfer until the service is restored, requiring temporary local storage, and more bandwidth to allow the additional buffer upload.  Most cloud service providers overlook this critical feature.


Inadequate Bandwidths Impact on Policy

Since most police agencies cannot afford the broadband connectivity required for the cloud, and they cannot afford to store everything in the cloud, they have to compromise policy.

Rather than recording all of the events that both the public and the police believe is appropriate, police opt to record less, record at lower resolution, record at lower frame rate, or store for a shorter period of time.

Using local storage (or maybe a hybrid model which only puts a subset of critical video files onto the cloud) allows you to store everything you should without compromise. Why bother with body worn cameras if you don't actually use them the way they are intended to be used?


Searching, Viewing, Redacting and Exporting the Data After Capture

Don’t forget that once your video is stored, you will need to quickly access it, search it, append it, redact it, and export it.  So, you should also evaluate how much bandwidth this will require, as well as what outbound data charges your service provider may impose.  Often vendors charge for outbound bandwidth.  Depending on your application architecture this can be significant.


End of Contract Blues

One of the unmentioned problems encountered by police agencies is moving their video data at the end of their service contract.  At the time that a contract is negotiated vendors will extoll the virtues of their proprietary video format for storage which improves compression, reduces storage requirements, etc. This may be true... until you need to extract your data from their repository.

The conversion of one file format and compression technology to another is called transcoding. Unfortunately this process can break the chain of custody for video files.  So, at the end of your contract you may need to keep using the vendors file tools in order to view the evidence.  Or you may be prevented from importing your video to a competing storage technology.  Of course you don’t find this out until after it is too late and all of your data is in a proprietary format.


Plan for Success

When planning a body camera architecture first agree on your policy.  Make sure you understand how it will be implemented as procedures.  Then determine how much storage will be required based on the policy requirements.  Look at the total long term costs over a three to five year period.  Be sure to allow for broadband connectivity costs, buffering hardware at your locations, cost of download and transcoding at the end of your contract.  Then make a decision whether you want to store your data locally or in the cloud.

Understanding how technology may constrain your policy will save you headaches now and in the future.


Shameless Plug

At HauteSpot Networks, our complete body worn video solution allows you to choose the right system architecture to meet your needs. Our HauteVIEW 100 camera is one of the best dockable cameras on the market today with industry leading features which are not found on many competing products.

The HauteSpot Dock Server is a simple, reliable and cost effective platform to which the HauteVIEW 100 cameras can download, process and store their video simply and easily. It can be ordered with hard disk storage, tape storage, or removable disk storage to accommodate whatever local storage requirements you may have. And the HauteSpot Dock Server can be used with either one of our two different evidence management software systems:

  • HauteVIEW Docking System software is a stand alone software solutions which runs on the HauteVIEW Dock Server and handles all of your basic transfer and storage functions right on the dock itself or to user provided NAS storage. It is a low cost solution which is perfect for smaller agencies on a limited budget.
  • HauteSpot Evidence Case Manager is a comprehensive evidence management system with is designed to work with multiple HauteVIEW Dock Servers, disk storage arrays, tape library systems and private or hosted cloud storage. Evidence Case Manager transfers video from cameras to the dock, from the dock to long term storage, provides robust searching, appending of additional meta data, and automatic redaction. It can be also used with third party systems like Computer Aided Dispatch, Interview Room Recording Systems, Records Management Systems and even Video Management Systems. Evidence Case Manager is your one comprehensive solution for all your evidence. Your officers will like the simple to use interface. Your IT manager will like the robust management features. And your Chief will like the low total cost of ownership.

And just wait until we release our new wireless, always connected HauteVIEW 200 cameras!

No comments:

Post a Comment