Bob's Adventures in Wireless and Video Headline Animator

Monday, May 31, 2021

Uncle Mac of The Greatest Generation

When I was a kid my dad had a best friend from his early days as a CPA, Lamont G. Mac Donald, or as I knew him "Uncle Mac". My dad and Uncle Mac were both returning from WWII and went to college on the GI Bill to get accounting degrees. My dad served in the Navy as an Ensign on an LST in the Pacific. You can see more about my dad in an earlier blog post. Uncle Mac was a Captain the Army in Europe.

This is the story of Uncle Mac. I never knew about any of this until Uncle Mac died at the age of 86 and we went to his funeral at San Joaquin National Cemetery in Santa Nella, when a man I had known for 50 years ended up being exposed as the greatest hero I have ever known.

Let me apologize for not having any photos of Uncle Mac. I probably have some in storage somewhere. So the photos attached are for example, not actually him. Also, I only have my fading memory to recall the citation which was read at Uncle Mac's funeral as to the events that lead to his Silver Star for valor and Purple Heart for injuries sustained. I have requested his service record from the National Archives, but not sure what I will get.

Lamont G. MacDonald "Mac" was born on June 23, 1921, in Kanab, Utah near St George. He was raised by his father Graham and mother Emily in a Mormon family on his family ranch, which stayed in their family through his passing. He had 5 brothers and 3 sisters. I don't know much about his family.

There was an interesting reference to him working in the suppers as a grocery clerk and a chauffer for movie executives filming in Kanab before joining the army. He went to Kanab High School and to BAC College for three years. There is also mention that he went on a 2 year mission to Texas for the Mormon church.

He enlisted in the Army in June 1942. He went to Camp Callan near San Diego for basic training, and then to Camp Davis North Carolina for Officer Specialist School, and finally Camp Benning GA for Infantry training. He was first assigned to the Coast Artillery and then after Infantry School transferred to the 289th Infantry Regiment, 75th Division. He was assigned a platoon as their 2nd Lieutenant. 

In June 1944 he and his unit flew to England, moved down to Wales, then sailed across the English Channel to land in Le Havre France, only to march up to Belgium where they were to start their combat with the Germans as "Green Soldiers". They had little experience.

It was now December and the 289th was part of an offensive with the Germans that was scheduled to happen on Christmas Eve. 

This web site describes the macro situation in pretty good detail. 

And this time line shows the overall view of the events leading up to that night.

Battle of the Bulge Timeline

11 Dec 1944 Adolf Hitler held a meeting with top German military commanders at the Adlerhorst headquarters in Wetterau, Germany, stressing the importance of the upcoming Ardennes Offensive.
16 Dec 1944 German troops launched Operation Wacht am Rhein, crossing the German border toward Belgium, opening the Battle of the Bulge.
16 Dec 1944 A German officer carrying several copies of Operation Greif (the codename for Otto Skorzeny's infiltration of "fake Americans" to cause confusion ahead of the Ardennes Offensive) was taken prisoner and the treacherous plan was revealed.
17 Dec 1944 150 prisoners of war of US 285th Field Artillery Observation Battalion were massacred by Waffen-SS forces at Malmédy, Belgium. Only 43 survived.
18 Dec 1944 The German offensive in the Ardennes Forest in Belgium began to stall after Americans began to fight back.
19 Dec 1944 Germans captured 9,000 surrounded US troops in the Schnee Eifel region on the Belgian-German border. Meanwhile, the US 101st Airborne of the Allied reserves and 10th Armored Divisions of the US Third Army were sent to Bastogne to hold the vital road junction in Belgium.
20 Dec 1944 Armored elements of German 6.SS-Panzerarmee captured Stavelot, Belgium, capturing the US fuel supply stored there for their own use.
21 Dec 1944 US forces captured Stavelot, Belgium, while the Germans surrounded Bastogne and captured St. Vith.
22 Dec 1944 In Bastogne, Belgium, the German surrender demand is rebuffed by General McAuliffe with the famous response "Nuts!"; meanwhile, the US Third Army shifted its axis of advance in attempt to relieve Bastogne. In Germany, Rundstedt suggested a tactical withdrawal, but the suggestion was refused by Hitler.
25 Dec 1944 US 2nd Armored Division, with British help, stopped German 2.Panzer Division just 4 miles from the Meuse River in Belgium.
25 Dec 1944 A surprise Luftwaffe attack on Bastogne, Belgium bombed Anthony McAuliffe's headquarters and the 10th Armored aid station. The three–storey building collapsed on top of the wounded patients and set the ruins on fire. Nurse Renée Lemaire was killed together with twenty-five seriously wounded patients, burnt to death in their beds. Soldiers rushing to pull away debris found themselves also machine gunned by the low-flying bombers.
26 Dec 1944 US Third Army under George Patton relieved the besieged city of Bastogne, Belgium.
27 Dec 1944 US troops began pushing German troops back in the Ardennes region, thus ending the German offensive.
28 Dec 1944 American troops began gaining ground in their counteroffensive in the Battle of the Bulge. Adolf Hitler ordered renewed offensives in Alsace and Ardennes regions against the advice of his generals.
30 Dec 1944 Germans again attacked in the Bastogne corridor in Belgium. Meanwhile, British troops attacked Houffalize, Belgium, but they were stopped by fierce German defense.
31 Dec 1944 US troops re-captured Rochefort, Belgium, while the US Third Army began an offensive from Bastogne.
1 Jan 1945 German troops began a withdrawal from the Ardennes Forest in the Belgian-German border region. Meanwhile, in retaliation for the Malmedy massacre, US troops massacred 30 SS prisoners at Chenogne, Belgium. In the air, the German Luftwaffe launched Unternehmen Bodenplatte, which consisted of 800 aircraft conducting low-level strikes against snow-bound Allied airfields in the Netherlands and Belgium. They destroyed 220 aircraft, mainly on the ground, but lost 188 aircraft of their own, as well as many experienced pilots who could not be replaced. This operation failed to achieve its goal of wiping out Allied air power based in the region.
3 Jan 1945 US First Army launched an attack on the northern flank of the Ardennes bulge in Belgium. Meanwhile, 1,100 Allied bombers, escorted by 11 fighter groups, bombed railroad and communications centers in western Germany.
5 Jan 1945 The German attack on Bastogne, Belgium was called off.
9 Jan 1945 US Third Army attacked towards Houffalize, Belgium, on the southern flank of the Ardennes bulge.
11 Jan 1945 British forces captured La Roche-en-Ardenne, Belgium, northwest of Bastogne.
12 Jan 1945 The Operation Nordwind offensive into France was finally stopped just 13 miles from Strasbourg. In Belgium, north of Bastogne, US and British forces linked up near La Roche-en-Ardenne.
13 Jan 1945 US First Army attacked near Stavelot and Malmédy in Belgium.
16 Jan 1945 US First and Third Armies linked up near Houffalize, Belgium, while British Second Army attacked near Maas River. The Germans were pushed back to the line prior to the launch of the Ardennes Offensive.
28 Jan 1945 The Ardennes bulge was finally pushed back to its original lines, thus ending the Battle of the Bulge.

Christmas Eve into Christmas Day

On Christmas Day, Company K, 290th, of which Uncle Mac's Platoon was a part, supported on the flanks by Cos. I and L, made a direct assault on a high hill controlling the approach to Hampteau. 

From his Silver Star citation which was read at his funeral: "There was a German machine gun nest dug in on top of the hill. Two soldiers from Captain (later awarded) Mac Donald's platoon, were assigned to storm the position and overtake it using grenades, while supported by cover fire from the rest of the platoon. The two soldiers were both wounded and fell to the ground. Also several of the platoon had been injured or killed. 

Captain Mac Donald first crawled on his belly out of his trench, to the wounded soldiers and dragged them back to safety, then he returned to crawl up to the German machine gun position by himself, throwing a grenade into it, and then charging into the bunker, killing all of the machine gun crew in hand to hand combat. During this assault he sustained significant injuries."

I don't know what his injuries were, but he received a Purple Heart for them.

Although pinned down by withering machine gun and mortar fire, these units seized enemy positions, thus ending the threat to Hotton. The high water mark of the German drive on Liege had been reached.

For his valor in combat he received the Silver Star. The third highest honor a soldier can receive for actions in combat. He saved the lives of two of the soldiers under his command. He eliminated a German stronghold single handedly, and he enabled the US Army to win the battle, ultimately leading to the defeat of German, and the liberation of Europe.

The Uncle Mac I Knew

After the war, Uncle Mac graduated from Utah State University and Golden Gate College. He went to work for LH Penny, a CPA firm where he met my dad. My dad and Uncle Mac worked together for many years and became very close friends. They both married and adopted families. My two sisters and I would travel with my parents from our home in the Bay Area to their home in Fresno often. And we went on vacations together to Blue Lakes, Donner Lake, and once to Hawaii.

For many years Uncle Mac managed the Cadillac-Olds dealership in Fresno, later opening his own auto leasing business. In the automobile business, he was well known for his honesty and integrity.

Uncle Mac always was a great husband and father. Once he was hand excavating a swimming pool in their new home. He dug out a set of trenches, covered them with plywood and dirt, and made an underground fort for us to play in, before ultimately digging the rest of the pool hole out. He was that kind of a father.

His wife, Bettie (Elisabeth) was an author who did very well in her own right. You can find her books on Amazon, well reviewed and still available.

Bettie and Mac's two children Brian and Karen were a great childhood friends. Sadly Brian ended his life in a very sad way before either of his parents passed.

In all the 50 years I knew him, I never once heard Uncle Mac talk about the war, his role in the most decisive battle in Europe, his injuries, or his Silver Star. The greatest generation never talked about their valor, or their trauma, they just stoically endured.

I miss Uncle Mac, Brian, Karen, Aunt Bettie and the golden days of childhood that they provided to me, along with my parents. A childhood of freedom and enjoyment secured by my Uncle Mac, a great American hero!

Thursday, April 28, 2016

The Hidden Costs of Body Worn Cameras…

Citizens, politicians, civil rights advocates, prosecutors, and police officers want body worn cameras (BWC's). They are an effective tool for law enforcement, providing reliable evidence and deterring bad behavior by both the public and the police.  However, deploying cameras and implementing an effective policy can be difficult both politically and technologically.


A little background

The current generation of body worn cameras function basically like any video camera:  someone has to push the button to turn the camera on and off.  That someone is the police officer who wears the camera.  The video is stored on an SD Card or internal flash, and then downloaded by the officer at the end of his/her shift to a storage repository.  We refer to this as the “dockable” camera model.
In a later article we will address the benefits of next-generation wireless (LTE/Wi-Fi) cameras which are always connected and how they change everything, but for now we will focus on dockable cameras.

The difficulty with body worn cameras is striking the balance between privacy rights, cost of operations, and the collection of meaningful evidence.  Legislators, police departments, police officer unions, and other interest groups all try to draft policy that strike this balance. Generally the result is a long list of rules dictating when the officer needs to turn the camera on and off.
In most cases the resulting policy is a pragmatic balance that allows police to record most interactions with the public where evidence needs to be collected, and where the camera is off when privacy is a concern or when nothing relevant is occurring. 

A typical policy would have the officer activate the camera when they are dispatched for an incident, when they believe a crime or infraction may be in progress, or when they believe a life or property threatening incident is occurring.  It would also have the camera turned off when officers enter a private property without a warrant or without the prior permission of the resident or property owner; when a sexual crime has occurred and the victim requests privacy; when a minor is being interviewed in the absence of parental consent; and similar situations. The officer would also have discretion to turn the camera off for his/her personal privacy.  Effective policy implementation relies on officer discretion.

Then there is the issue of retention, which generally keeps all recorded video for a minimum 30-60 days, with incident-relevant video kept for longer periods, from 1 year to forever.

To oversimplify the positions of various interest groups1 :

  • Privacy advocates want the least amount of recording possible, only recording evidence of actual crimes where only the perpetrator and individuals directly involved are in the field of view. They want the video retained for the minimum time possible, with complete redaction before being released.
  • Social Justice groups want only recording of police actions, exclusive of the public, unless there is evidence which would exonerate someone.  They want evidence of police wrongdoing kept forever, and the rest immediately deleted.
  • Police officer unions want to record only criminal evidence, leaving out anything which might be taken out of context and result in disciplinary actions. Once the video is recorded, police officers  want the right to edit it, then determine how long to keep whatever they think is relevant.
  • Prosecutors would like everything recorded, giving them the ability to select what evidence becomes part of the record.  They would like to keep everything, but only release what they decide is relevant.
  • Courts prefer continuous recording of everything. And everything should be kept forever.
  • News media wants recording of all sensational events, with no redaction. They would keep the video themselves forever once it is released.
  • IT departments and city budget managers would like to record a fixed, predictable, and minimal amount of video, in order to reduce infrastructure costs. They actually drive the pragmatism of the policy.
  • Service providers want to record everything possible and store it forever.

The results of these divergent interests on policy vary dramatically, from a few minutes of video per shift on one extreme to hours of video on the other.


Mathematical Limitations

Let’s assume that all of the interest groups compromise and arrive at a policy right in the middle with the result:

  • 2 hours of video per officer per 10 hour shift is recorded.
  • Cameras record at the relatively low resolution of 720p (1280x720 pixels) at 30 frames per second.
  • The resulting video files are about 2.472 Gigabytes per hour, or 4.944 Gigabytes per shift.
  • All video is saved for 30 days.
  • 6% of all video is deemed to be potential evidence and is retained for 1 year.
  • 2% of all video becomes part of a case file and must be retained for 5 years.

Then let’s use an example of a mid-size agency with 60 officers who work 3 shifts per day with 20 officers per shift, each officer with his/her own camera. (Yes, the officers get no time off in my example):

  • 20 officers per shift X 2 hours of video per shift X 2.472GB per hour of video X 3 shifts per day = 297GB of video per day. This is how much needs to be stored and/or transferred to the cloud each day.
  • 30 days per month of all video = 8.7 Terabytes of video per month (rolling)
  • 6% of video per month is potential evidence = .06 X 8.7 TB = .522TB per month
  • 1 year Long term storage of potential evidence = .522TB x 12 months = 6.264TB per year (rolling)
  • 2% of video per month becomes evidence = .02 X 8.7TB = .174TB per month
  • 5 year Long term storage of evidence files = .174TB x 60 months = 10TB per five years (rolling)

Once you have recorded this video, you have to store it, protect it as evidence following chain of custody rules, make it searchable, and provide it to various constituencies in either raw or redacted from to protect privacy.

Regardless of how you store your video, you will need to purchase the cameras, which cost from $400 to $1,000, depending on the supplier. Then you will need a docking station, which can cost an additional $50 to $200 per port depending on the supplier.  These costs are fixed for both local storage or cloud storage architectures.

This is where a decision has to be made:  do you buy local storage, or do you use a vendor solution in the Cloud.


Local Storage

The math to determine the cost of local storage is pretty simple.  You need to buy a server, disks and tape, provide power, keep it maintained, and provide support.  It needs to be secured, following CJIS2 standards (for both physical and network security).   

While these may seem onerous at first, they are not unlike traditional evidence chain of custody management. Not transiting public networks or using a hosted facility means greatly simplifying the implementation of a complaint facility.

Moving the video from the camera to a dock to a storage system on site is straight forward. Everything can be on the same LAN and is local, so you have plenty of bandwidth.

This could be covered with a single PC Server with 12TB of RAID disk storage, an LTO (Long Term Offline) tape drive, 4-2TB tapes, and software supporting transfer, storage, search and redaction management.  Let’s say this whole system costs $40,000 in capital and about $5000 a year in recurring software and maintenance. Over 5 years the total cost of ownership is about $65,000. Allocating this to each camera it works out to be about $1083 per camera, or about $18 a month.

Of course you need an IT support engineer to maintain the system.  But this will not be a full time job and in most cases can be provided through existing resources.

You need to add in the cost of power and air conditioning as well, which could run about $100 a month.
No other infrastructure is required.



Cloud means using shared servers hosted at a remote facility.  This can reduce the labor and support requirements from IT, although not completely eliminate it, as the cloud service provider maintains the servers, storage, and support infrastructure at their facility.  For small agencies this may seem to be a very attractive option.

Your end to end network, reaching from your facility to the cloud hosting facility, must be CJIS compliant.  At present there are very few such facilities.  

Pricing for cloud storage solutions are advertised from about $79 per month per camera with unlimited storage.  But there are typically additional service fees which can add up to well over $100 per month per camera.  This adds up to $6,000 per camera over 5 years.  Compare this to buying your own storage solution at $1,083 per camera – what would be more affordable to you?

In some cases vendors set a maximum storage allocation per month which is covered by their base service cost.  Once this allocation is exceeded, additional costs (from 6.5 to 12 cents per gigabyte per month) are added.  A number of agencies have been surprised by these charges.  In some cases these charges are waived in the first year and then become payable after the agency has built a large repository of evidence.


How Secure Is your Data?

The CJIS guidelines provide great guidance on implementing security for video evidence which transits the public Internet, exists in shared storage environments and leaves the custody of the agency which created the video.  There is no doubt that these guidelines are well thought through.

However, as we all know, hackers seem to find ways into just about every facility, and CJIS data centers represent an attractive honeypot of data for them.  With all government agencies sharing one massive data center, it is incredibly attractive.  Once one server is compromised, the entire facility is at risk.

Further, the legal admissibility of cloud stored video evidence is still an open topic.  Often, the physical location where the data resides may not be known.  It may have been moved from one jurisdiction to another even though it never left the service provider's network.  Challenges to data access, data tampering, etc. have not yet been vetted through the courts. 

Keeping data local eliminates many of these issues, provided due care has been taken to physically and logically protect evidence files.  A well-engineered local storage system which follows CJIS guidelines and is kept off the Internet (air gapped) may avoid much of the current risk of cloud storage.

This is not to say that cloud storage is any more or less secure than local storage, but there are many unresolved issues with cloud that will only come to closure over time.  It may be better to wait on cloud.


Broadband Bottleneck

A basic assumption to use cloud services is that adequate broadband connectivity exists between the police facility and the cloud facility.  This is where police agencies can get bitten, hard.

Let’s go back to our storage requirement of 297GB of video per day.  A transfer of this video evenly throughout the day equals 12.375GB of video per hour.  To do this you will need a dedicated upload speed to the Cloud service provider of 33Mbps3.  That is UPLOAD speed.  It needs to be DEDICATED, not shared.  And that is persistent, running 24 hours a day, 7 days a week, 365 days a year.

Most broadband connectivity is asymmetric with the download speed being the focus. Upload speeds are typically much lower.  Higher upload speed requires the purchase of high cost business services such as 1Gbps Ethernet.

Based on the most recent FCC Measuring Broadband America4 report for 2015, the mean upload speed for consumer services is about 5Mbps:

Click to enlarge
So generally, you will need to have an Ethernet Virtual Private Line (EVPL) service or a dedicated Metro Ethernet connection between the police facility where the download docks are located and the cloud storage provider location. This type of service can cost thousands of dollars a month and can take months to years to install depending on your location.  This cost has to be added into the calculation of your cloud solution.

The other consideration is that you need to be transferring video continuously.  If there is an interruption in service, you will need to buffer your transfer until the service is restored, requiring temporary local storage, and more bandwidth to allow the additional buffer upload.  Most cloud service providers overlook this critical feature.


Inadequate Bandwidths Impact on Policy

Since most police agencies cannot afford the broadband connectivity required for the cloud, and they cannot afford to store everything in the cloud, they have to compromise policy.

Rather than recording all of the events that both the public and the police believe is appropriate, police opt to record less, record at lower resolution, record at lower frame rate, or store for a shorter period of time.

Using local storage (or maybe a hybrid model which only puts a subset of critical video files onto the cloud) allows you to store everything you should without compromise. Why bother with body worn cameras if you don't actually use them the way they are intended to be used?


Searching, Viewing, Redacting and Exporting the Data After Capture

Don’t forget that once your video is stored, you will need to quickly access it, search it, append it, redact it, and export it.  So, you should also evaluate how much bandwidth this will require, as well as what outbound data charges your service provider may impose.  Often vendors charge for outbound bandwidth.  Depending on your application architecture this can be significant.


End of Contract Blues

One of the unmentioned problems encountered by police agencies is moving their video data at the end of their service contract.  At the time that a contract is negotiated vendors will extoll the virtues of their proprietary video format for storage which improves compression, reduces storage requirements, etc. This may be true... until you need to extract your data from their repository.

The conversion of one file format and compression technology to another is called transcoding. Unfortunately this process can break the chain of custody for video files.  So, at the end of your contract you may need to keep using the vendors file tools in order to view the evidence.  Or you may be prevented from importing your video to a competing storage technology.  Of course you don’t find this out until after it is too late and all of your data is in a proprietary format.


Plan for Success

When planning a body camera architecture first agree on your policy.  Make sure you understand how it will be implemented as procedures.  Then determine how much storage will be required based on the policy requirements.  Look at the total long term costs over a three to five year period.  Be sure to allow for broadband connectivity costs, buffering hardware at your locations, cost of download and transcoding at the end of your contract.  Then make a decision whether you want to store your data locally or in the cloud.

Understanding how technology may constrain your policy will save you headaches now and in the future.


Shameless Plug

At HauteSpot Networks, our complete body worn video solution allows you to choose the right system architecture to meet your needs. Our HauteVIEW 100 camera is one of the best dockable cameras on the market today with industry leading features which are not found on many competing products.

The HauteSpot Dock Server is a simple, reliable and cost effective platform to which the HauteVIEW 100 cameras can download, process and store their video simply and easily. It can be ordered with hard disk storage, tape storage, or removable disk storage to accommodate whatever local storage requirements you may have. And the HauteSpot Dock Server can be used with either one of our two different evidence management software systems:

  • HauteVIEW Docking System software is a stand alone software solutions which runs on the HauteVIEW Dock Server and handles all of your basic transfer and storage functions right on the dock itself or to user provided NAS storage. It is a low cost solution which is perfect for smaller agencies on a limited budget.
  • HauteSpot Evidence Case Manager is a comprehensive evidence management system with is designed to work with multiple HauteVIEW Dock Servers, disk storage arrays, tape library systems and private or hosted cloud storage. Evidence Case Manager transfers video from cameras to the dock, from the dock to long term storage, provides robust searching, appending of additional meta data, and automatic redaction. It can be also used with third party systems like Computer Aided Dispatch, Interview Room Recording Systems, Records Management Systems and even Video Management Systems. Evidence Case Manager is your one comprehensive solution for all your evidence. Your officers will like the simple to use interface. Your IT manager will like the robust management features. And your Chief will like the low total cost of ownership.

And just wait until we release our new wireless, always connected HauteVIEW 200 cameras!

Friday, December 25, 2015

A Story of Giving

Back in the early 1990s I lived in Sri Lanka as the General Manager for American President Lines. It was an adventurous time for me and there were many things learned. But one of the most memorable was the friendship which I built with an unassuming man who worked at the Horton Plains National Park named Kulasuriya.

The Horton Plains is a truly amazing place of unbelievable beauty. It is filled with unusual animals such as white monkeys, an endless variety of colorful birds, deer, snakes, and an occasional leech.

On one trip up to see the park we happened to come across a short man sitting on the side of the trail watching monkeys in the trees and drawing with a number 2 pencil on some old scrap paper. We stopped to watch him work and he really was pretty good. His hands were hard and caked with dirt and his tools were really poor. His pencil was sharpened with a knife and was whittled down to not much more than a stub. And his paper was stained and torn. It was clear he was using the discarded trash from the rangers to make his art.

We stopped and sat down next to him and started to talk. We learned that he lived in the rain forest and that he liked to draw when he was not working. But there was more to him.

The park is guarded, and trails maintained by rangers who live year round in a small cabin near the entrance to the park. Of course there is someone who has to clean and maintain the cabin for the rangers. This job has the title of Peon, and is the lowest caste of jobs in Sri Lanka, which has a similar, although less rigidly enforced, caste system like India. Kulisuriya was the Peon of the ranger station. He was the lowest of the low economically. 

Kulasuriya was born and raised in Moratuwa, south of the capital of Sri Lanka, Colombo, where I lived. His father has passed away some time ago, so Kulisuriya was responsible for caring for his mother and younger sister. His salary of about $30 a month was what kept the family alive.

On a good day with light traffic it was about 90 minutes by car to drive there from my house. On the bus, it might take 2.5 hours. From Colombo up to Horton Plains was another story. By car it could take 4 to 5 hours. But by bus it could take 8 or 9 hours. Kulasuriya would travel back and forth from Moratuwa to Horton Plains and back once a month so he could check in on his mother and sister, make repairs to their meager home, and enjoy two days off, his only non-working time.

Kulasuriya and I talked for about an hour and he told me about his aspirations to become a wildlife artist. However he needed to care for his family and buying proper drawing materials was well beyond his means.

On my next trip home to California, I stopped in at an art supply store and purchased some colored drawing pencils, a clipboard, and a few packages of drawing paper. It was a small gesture, but I thought that maybe Kulasuriya might be able to get started with some proper supplies and maybe sell some of his drawings to other tourists in order to start making a living. When I returned to Colombo I sent the supplies up to the rangers station so Kulasuriya could use them.

I forgot about him for a few weeks.

Then one day Kulasuriya showed up at my house in Colombo. He had traveled all the way up from Moratuwa to Colombo by bus in order to thank me and deliver to me a small gift. I was very surprised, but delighted to see him. However my housekeeper and driver both looked at me like I was crazy for letting this poor little man in the garage, let alone into my house. They were shocked that I would allow a lowly peon anywhere near my home. I looked past this and invited Kulasuriya in.

He gave me a small package, wrapped in newsprint. He told me it was something for me. I unwrapped it and was surprised to see a small ceramic bunny rabbit. It was not particularly beautiful. There was nothing especially unique about it. It could be purchased at any shop for maybe 30 rupees ( $1). But coming from Kulasuriya, it was amazing. Here was a man who had to feed three people on $30 a month. A man who traveled 8 hours on a bus so he could see his mother and sister once a month. A man who had nothing.

I could not let this go, so I offered to give him a ride back to Moratuwa in my car, rather than have him endure another 2.5 hour bus trip home after coming all the way up to see me. Believe me, taking a bus in Sri Lanka is nothing like the air conditioned, comfortable metro transit that we have in the USA.

My driver, Fernando, was furious that I was going to make him drive Kulasuriya, a peon, and I all of that way. He tried to make excuses. I said I would drive him myself. Fernando relented, realizing that my driving in Sri Lanka, at night, was really dangerous. This was the time of the LTTE, military roadblocks everywhere, and extortion of foreign nationals who got into traffic accidents was very common.

So we set off into the dark wet night to deliver Kulisuriya home.

Home for Kulisuriya was a small, one room corrugated metal shack with a dirt floor, a single wood chair, a small table, three bamboo reed mats to sleep on, a dresser, a small cabinet, and a single light bulb hanging from the ceiling. There was no radio, no tv,  no wall coverings, no curtains. Plastic sheets hung over the windows. Cooking was done over a fire in the corner of the shack, ringed with stones. Outside was a round well, with a rope and a bucket. I didn't ask where the toilet was. Hopefully it was not next to the well.

Kulisuriya invited me in and asked me to sit in the chair. He introduced me to his mother and sister, neither of whom could speak English, and my Sinhalese was pretty bad. But I could see how excited they were that an American business man was in their house, invited by their brother. I also noticed that the neighbors were all watching and whispering amongst themselves.

Kulisuriya then opened the cabinet and I could see a single cup on the shelf. I don't know how the family drank otherwise. But it was a clear glass tea cup. And next to it was a small tea pot. A sense of dread came over me, as I realized that he was going to make a cup of tea for me. He asked me if I would like some tea, and of course I had to accept. His mother went outside to the well and got a bucket of water. She brought it in and filled up a pot on the fire and we waited, idly chatting, while the water boiled. At least I hoped it was boiling.

After what seemed like an eternity, Kulisuriya poured the water into the teapot and added some loose leaf tea. We waited a few more minutes for the tea to steep, and then he poured it into the glass cup on the table in front of me. The water was brown and cloudy, and I tried to see if there was anything floating in it that did not look like tea leaves. I smiled a weak smile and politely took a small sip. He asked me how it was, and I said "delicious".

He then went back to the cabinet and returned with a package of cookies. Vanilla cookies is what the label said. My guess is that the cookies had been purchased some years ago for an honored guest that maybe did not arrive. The label was old and faded, and the price tag on the package was peeling off, as the glue dried up. That package of cookies must have cost them another 30 or 40 rupees, and I really did not want them to squander their treasure on me. But they insisted. Kulasuriya opened the package and placed two cookies on a plate, also extracted from the cabinet. He set them down in front of me and suggested that I eat.

I ate both of the cookies. The dry, stale, hard, crumbly cookies that had been sitting on that shelf for years. The treasure of a family who had nothing. They were offering it to me, in this moment. It was everything they had. Everything they valued. Everything that they could offer to a friend from a distant land whom they might not every meet again.

I was stunned in the moment. I was saddened that all I had done was buy them some pencils and paper. I became keenly aware of my own life of relative ease and abundance. It was almost like I was at communion in church. It awakened something in me about thankfulness, about gratitude. But also about pride and self sufficiency. Kulasuriya was not asking for anything from me. He never did. He explained with pride what he did to support his family. He offered these gifts to me, not with the expectation of repayment. He was giving me the most valuable thing he could. He showed me kindness and gratitude.

On the ride back home I had to stop and vomit several times and I was as sick as a dog for the next week or so. The price of my experience with my friend.

Kulasuriya mailed me some of his first color drawings done with his new pencils and paper. I bought him a more extensive drawing kit which included more pencils, pens, and acrylics.

Hostilities in Sri Lanka heated up. My daughter Zoe was conceived, and it was time to leave Colombo. I lost touch with Kulasuriya, but I still have the little ceramic bunny on the mantel over our fireplace. Whenever I see that little bunny I think about was giving really means, and how a little man with nothing touched my life.

When I get home I will update this blog with a photo of the bunny.

Thursday, February 26, 2015

There is Nothing Neutral in Title II Regulation of the Internet

It has been a long time since I posted something to my blog. But Title II regulation of the Internet in the guise of implementing Net Neutrality is such as big issue I just had to write something.

A Little Background

I started working with FidoNet back in about 1979 in high school and was enamored with CompuServe throughout college. I joined the Well and used my Hayes 1200 baud modem to connect to others to discuss technology, social issues and the like. But I knew that there was going to be a commercial aspect to this that extended beyond just being a substitute for telephone party lines. 
In about 1988 I started a TCP/IP based BBS system in Oakland I called TransTech. Over time I hooked up to NSFNET. I was given a class C network (which I would later have to give back). And finally I found my calling when I built one of the first wireless ISPs in the Bay Area providing coverage off the top of the 1200 Broadway building in Oakland to the newly decommissioned Alameda Naval Air Station.

I never once had to get a license, ask permission, adhere to standards of decency, or worry about what my customers were using their connectivity for. I provided a pipe through which data flowed. I paid upstream providers for larger pipes, and I never had time to worry about metering content. I knew that if I did meter content, there were lots of competitors ready to take my place. Pricing was set by the market. Service was set by the market. Customers chose the best service for the best price.

Fast forward to present. Pretty much the model of operation for the Internet remains the same. 
However, fear of something that never even has been a real issue prompted cries for government intervention in order to enforce Net Neutrality. But people forget that we live in a Laissez-faire economic system in which transactions between private parties are relatively free from government interference such as these massive Title II regulations. The market will punish those that filter traffic through competition. 

Wireless and over the top/virtual ISPs exist as a locally owned bypass around monopolistic cable and telco companies. This keeps them in check. But not for much longer. We have now empowered the government to regulate from top to bottom a utility which was functioning perfectly well with no regulation. Beware unintended consequences.

Net neutrality and Title II FCC Internet regulation are not the same.

Net neutrality is the concept of treating all packets equally, regardless of the source or the content. But from a practical perspective this is not realistic. There is limited bandwidth on networks. Traffic needs to be managed, prioritized in order for it to all get through. Every network manager will tell you that they apply QoS (quality of service) prioritization on their network. You have to, unless you have limitless capacity. Network operators should be free to define these rules. And customers should be free to choose which networks they want to use based on the quality of service they receive. Networks will evolve based on the type of QoS prioritization they apply. Customers will have choice in a free market.

Net neutrality can and is achieved through competition. A free and open market is what is needed. Maybe there could be some work on local right of ways in order to spur competition. But Title II is like putting out a match with a fire hose.

Title II is an antiquated set of government restrictions designed for the bygone telephone monopoly of 1937. Title II sets up regulatory bureaucracies overseeing a utility that wants to be free, not controlled. 
The first casualties of this regulation will be the local ISPs who provide the very competition to the large monopolies who the advocates of Title II were trying to reign in. So unintended consequence #1 is elimination of competition and choice.

The second unintended consequence will be taxation. Now that the government has control of the Internet it will want to wring out of it every last penny it can. Expect fees, licenses, use taxes, excise taxes, and the like to start appearing on your ISP bill shortly. And they will grow.

The third unintended consequence will be increases in the barriers to entry for using the Internet as a vehicle for free expression. With no license fees, no bureaucracy, no regulation, today anyone can set up a blog, a web site, or chat at a cost approaching zero. But the government will make it increasingly harder for people to do this.

The fourth unintended consequence will be censorship. The fairness doctrine. The seven deadly words. Hate speech, and other arbitrary rules will be applied to curtail the use of the "public" (read government regulated) Internet. 

Beyond this, I am sure their will be many other unintended consequences. Curtailment of innovation. Decline in infrastructure investment. Who knows.

Pirate Internet

There is one thing everyone can start planning to do. Buy an outdoor wireless router. Load it up with some mesh software like Freifunk Wireless, and create a local community network which is apart from the government run Internet. Using currently unlicensed radio frequencies, you can bypass the regulation which is coming. You can maintain your freedom. And you can create competition for the new Ma Bell or whatever we will call the super monopolies which this is sure to produce.

Tuesday, December 3, 2013

Carriers Move to Limit Public Safety Cellular Data Transfer

A customer of ours told us, and this has not yet been confirmed, that Verizon is moving to follow the T-Mobile model of cellular data transfer throttling for public safety. Basically you will pay for a 5GB transfer cap at full LTE 4G speed (measured in Mbps) at a fixed fee, then once you hit the cap you can still transfer but your speed will be reduced to 256kbps or less. I assume that you can still purchase a plan with a higher transfer cap, but it will cost you more.

Actually I use T-Mobile 4G with such a plan for my cell phone and it is a great deal. But then I am only surfing the net, checking my emails and checking in on Facebook.

For police and other public safety personnel who have become accustomed to unlimited transfer at full speed, this presents a big problem. Many of our folks in blue use IP cameras to stream video from a remote site over cellular. It is easy to install, it is available everywhere, and it is compact.

Unfortunately streaming video requires a lot of bandwidth. A typical 720p at 15fps camera will use about 1.7Mbps. That is about 13MB per minute, or 765MB per hour. So you can easily reach your 5GB transfer limit in about 6 1/2 hours. So if you need 24x7 streaming you have to compromise.

Many IP cameras support multiple streams for the same video and some have SD cards that allow up to 64GB of storage (83 hours of 720p at 15fps). So you can send one small stream over cellular at a low resolution and slow frame rate while storing locally. If you were willing to run at VGA resolution (640x480) at 4fps you could get an video stream for "situational awareness" in 150kbps. But that is not a very good image quality, it is slow and jerky, and it still will burn through your data plan at 1MB per minute, or 1.6GB per day. So you could run for about 3 days a month before you run out of data transfer on your plan.

Another issue with this strategy is that you will need to determine what the minimum data rate that your cellular link supports is. If you are in an area with only 3G and are on the fringe of the coverage area or in an area that is highly congested, you may only achieve 100kbps. So you need to set your camera at this low rate in order to assure that your stream gets across. Your camera cannot adapt.

And then you have to think about retrieving your video evidence from your camera. You have your cellular connection, but if you try to transfer the locally stored high quality video from the camera back over that cell connection, you are again faced with your data cap.

And what if you want a 1080p camera or a 5MP camera, or a 20MP camera, or faster frame rate. What will you do?


What you really need is a video storage and transmission system that was designed for the limitations of a cellular world.

  • A system which allows you to use the camera you want, at the location you want, regardless of what kind of power is available or how much space there is.
  • A system which can store weeks or even months worth of full frame rate, full resolution video on site and yet allow you to easily retrieve it over the air.
  • A system that can accept any USB modem from any carrier.
  • A system which maintains an outbound connection to your HQ location, but does not send video when no one is watching.
  • A system which automatically adjusts it's frame rate, resolution, and compression for the best possible live viewing given the bandwidth available.
  • A system which supports remote viewing using any device you have, be it a pc, iPhone or Android device.
  • A system with wide operating temperature and choice of storage media up to 1TB internally.
  • A system that supports cryptographic tunneling for security.
  • A system that gives you a choice of using a simple Windows remote desktop or web interface for management.
  • A system that is backed by a team of experts in IP wireless video.
  • And a system that is reasonably priced.

You want a HauteSpot Networks MVE system.

I could go on about all of the benefits of the MVE system, but the best way to decide for yourself is to try the system out. Call HauteSpot Networks to order a demo system. +1 (800) 541-5589

Thursday, May 16, 2013

The Power Paradox

Three years ago HauteSpot Networks introduced the microNVR which was and remains a valuable tool for video security. The premise of the microNVR is to be a small (as in "micro"), power efficient, computing device that has all sorts of connectivity and the ability to store and process video at the edge. In this case the edge means on a pole, in a car, along a fence, on a roof, out in the woods...anywhere you just cannot place a big computer.

For some time now we have been working towards a upgrade replacement for the microNVR. We need to increase it's video processing capability and its storage, while retaining the same general low power consumption and size.

Intel left us without a good roadmap. The Atom processor family has basically gone without an upgrade since 2009, while Intel focused its resources on building their Mobile Phone System Platforms, they let the Atom family languish. And in the Core product line there really were no viable options presented either. Most of their chips started at 25 watts or about 10x the power consumption of the Atom.

The Z550 Atom that we use in the microNVR runs at 2GHz and is a single core processor with two threads. The GMA500 GPU is capable of encoding about 30fps of 1080p video into H.264 streams. If you have 3 two megapixel cameras running at 10fps each, and are encoding the video or transcoding it for transmission, then you have pretty much consumed all of the capacity of the processor and GPU.

CPU Boss Shootout review...
I have no idea where they got their power number.
The Atom is 1/10 the power of the Fusion.
So we went over to AMD and experimented with their Fusion product family, which was presented as having great video processing capabilities with low power. What we found was that AMD had no better story than Intel. Their embedded processors started at about 17 watts and their GPU was designed for better performance, at least in theory, than the Intel Atom. The AMD T40N processor, which runs at 1GHz does have a better GPU for decode (like watching a movie). But that GPU only decodes certain codecs well. H.264 encode requires third party libraries (from a third party called Main Concept) that when we tried them did not work. We contacted AMD and they said that the GPU did not support H.264 encode but that later versions of the Fusion family would. So that meant that any video processing on the platform other than decode had to be done on the CPU in software, which, because the CPU was clocked at only 1 GHz was even worse than the ATOM.

So we waited, and waited, and waited. Finally Intel released a low power 3rd generation i3/i5/i7 processor family. The really good news is that these processors have Quick Sync technology built into them as part of the HD Graphics 4000 GPU. Quick Sync accelerates both encode and decode of H.264 (for applications that are designed to use it).

So we were excited to take a look. There is a lot of supporting documentation regarding the performance that Quick Sync brings to H.264. We expect that it will mean about 3x performance over previous generations of Intel Core processors. Or about 98fps of 1080p H.264 transcode with almost any member of the i3/i5/i7 3rd generation family (Ivy Bridge). However, running at 17W means that our system will consume somewhere around 23W versus the 10W that we consume today with the Atom system. This much power means twice as many solar panels, twice as many batteries, etc.

Then we got some even better news that Intel was going to release a new Atom processor in the second quarter of 2013 and that this new Atom (Bay Trail) would have Quick Sync in it and have the same HD 4000 GPU. Great, this is the CPU we want...well don't get too excited...

CPU World reported that Bay Trail was going to be delayed until mid 2014. This would have been a 3W chip that had all of the performance we wanted in the exact package size we need. But who knows when or if this will get released.

So we are going to punt. This probably means building a system for release in the next quarter based on the low power i3 or i7 processor. The system will be a little bit bigger and take more power. Then when the new low power Bay Trail comes out, we will jump back to that.

Unless Intel jumps around some more. Keep your fingers crossed.

Sunday, April 14, 2013

Getting through the hype at ISC West

It was interesting to walk the ISC West 2013 show floor and speak with vendors. There were two wild claims being made at this years show and misrepresentations of what vendors were showing that I think integrators need to be aware of.

Video Streaming Over 4G

Example of a matrix view of thumbnails on a VMS
The most outlandish claim came from one VMS vendor who was saying that their product could transmit 16 megapixel camera streams at 30fps and full resolution in 1Mbps of bandwidth over 3G/4G. This was just a flat our misrepresentation of what they were doing.

There is a huge difference between transporting 16 streams as full frame rate and full resolution and sending thumbnail views of camera streams. Let me explain.

A typical H.264 IP camera with a 1.3 megapixel resolution will compress and stream video in 25.6KB frame size. At 30fps, that is 768KBps or 6144Kbps or 6.1Mbps. This assumes some motion, etc, but is a general ballpark. By turning up compression, you can get this down to around 2.5Mbps at the cost of image quality (artifacts).

In a typical security configuration the cameras send a primary stream over a high speed local area network to the VMS server without concern for bandwidth, recording at best quality and frame rate. Sending a 2.5 Mbps, or even a 6Mbps stream, between the camera and the server is not usually an issue. However, sending from the server out to a remote client such as a smartphone over a narrow cellular link can be an issue.

Example of how HauteSpot MVE Connects
If you are using a LTE phone you may be lucky and get 8 Mbps or more down, depending on network conditions, but rarely is this sustainable over long periods. A more likely down stream speed is going to be around 1 to 2 Mbps.

If you tried to stream at full resolution, full frame rate and full image quality even one H.264 1.3 megapixel stream the video will have little chance of getting through in real time. You will need to buffer a long time (aka YouTube, Hulu, etc). Remember that most broadcast streaming is non-realtime and they use multipass encoding techniques to improve the quality and reduce the size of the streams they send.

Further, a 1080p monitor can display 1920x1080 pixels or approximately 2 megapixels at any given time. So sending any more data than this from the server to your client is really useless.

So what most VMS systems do is set a second stream from the camera to a lower resolution for live viewing. They are still recording the primary live stream at full resolution. When someone goes to the VMS from their cell phone, they first get a thumbnail of each of the cameras using the small substream and then, when you click on a camera to look at in detail, the VMS will shift from the small secondary stream to the primary full resolution stream. But even this may be scaled and transcoded on the server to fit the size of the screen of your device.

So, really what you are getting from the VMS to the smartphone client is a 2 megapixel or smaller transcoded rendering of your cameras. When you are looking at 16 cameras, you are not receiving all 16 streams at full resolution, you are receiving either a bunch of small substreams or a transcoded single stream representing all of the substreams in 2 megapixels. This may then additionally be compressed to fit into a 1 Mbps pipe. Think of it as a "image of your streams".

There are many VMS who do this and the technique has existed for years. It is a commonly known method that is well documented in the public domain.

What is missing from most vendors implementations are several critical elements which are addressed by the HauteSpot MVE system:
  • Inbound Mobile Ingest of Cameras - It is all well and good to be able to view your cameras remotely over 4G, but connecting your cameras to the VMS over cellular is a different story. This requires uplink speed, which rarely exceeds 500kbps and can be extremely variable. MVE dynamically adjusts the uplink speed to fit the available bandwidth. It is not preset to a low "live view" rate and size. If you have good bandwidth, you will get a better image.
  • Persistence - Most VMS have no ability to deal with the constant disconnects of cellular. Roaming between cell towers, changing IP addresses, and lost signal all contribute to loss of connection. MVE makes and keeps persistent connections that have been proven to work on cellular networks in the worst conditions (hurricane Irene, tropical storm Sandy, floods and hot weather).
  • Remote Record with Transfer - The HauteSpot microNVR allows remote recording at full frame rate and resolution at the camera source. It then provides a persistent connection to the MVE server and the dynamic rate adaptation which adjusts to changing available bandwidth. It also allows remote, chain of evidence transfer of the high resolution video from the remote site in batch.
There is a big difference between transmitting full resolution, full frame rate, full quality video over cellular and sending a transcoded or down-res version of video from megapixel sources. Know what you are getting.

VMS IP Video Decode Capacity Vs Record Capacity

As we walked the show we asked a simple question of a number of VMS vendors "How many 1080p 30fps live streams can you simultaneously display (assuming that they had a multiple monitor video card on their system)?" The answers ranged from 4 to 64. And the answers were coming from field application engineers, not sales people.

A VMS provides a number functions including recording video streams to disk, displaying live and recorded streams, providing search capabilities, etc. The answers that we got indicated that the vendors really did not understand the difference between storing video and decoding video for display.

Again, IP VMS systems receive incoming video streams from cameras. These are typically either H.264 or MJPEG streams that have a fixed frame rate, resolution and compression profile as set on the camera. Most VMS systems will receive the video stream and write it to disk. This function does not require any significant CPU or GPU functions, since it is really just network IO. The system is limited only by the bandwidth of the network as to how many streams it can support. Disk IO is generally faster than network IO.

Tom's Hardware Benchmark for H.264 encoding
Displaying a live or recorded stream does require CPU and/or GPU resources. A stream must be read and decoded for display. This is significant work for a computer.

The chart on the left shows just how much time it takes to encode 1080i video using hardware and software. This is an example of when you are reading a raw source and sending it to a client to view. Performance is similar for decoding. As you can see, using hardware such as the Intel Quick Sync technology built into Ivy Bridge based systems can significantly improve performance. Without Quick Sync, software encoding/decoding consumes approximately 25-50% of a quad core CPU for just one 1080p 30fps stream. Using high end video cards helps some, but again, they are targeted at single or maybe dual display systems, so they really don't try to decode more than what they are capable of displaying.

If you were to use the hardware acceleration of Quick Sync, then you could get 120 fps of 1080p
video encoded or decoded for playback. So the best possible case for displaying full 1080p at 30fps is 4 simultaneous streams. But this is using hardware. If you are using software to decode/encode, as most VMS systems do, then you are really down to about 1 stream. Which makes sense given that most VMS systems only have one display attached.

So the vendors that said they are able to simultaneously decode 12 or 64 streams are clearly not correct.

If you are writing a file to disk or reading a file from disk to and from the network, then yes, you can support many cameras. If you are trying to actually display the video, then the capacity is much much lower.

It is interesting that only a couple of products like Network Optix HD Witness or HauteSpot's MVE system take advantage of Intel Quick Sync hardware acceleration in order to increase performance and are designed for high definition video processing.

This is actually a good thing for companies like RGB Spectrum who make display wall processors that allow you to aggregate multiple server video outputs into a single consolidated display. Solutions like their QuadView HDx allow you to consolidate multiple video sources into one screen, so even if you can only get a single 1080p stream out of your VMS, you can combine it with other servers to create faster, full frame rate, full resolution systems. Watch for more on high resolution display, particularly 4K, when I review NAB 2013 in my next blog.