Video Analytics: Saving Lives In Real Time
by Security Electronics and Networks | @Articles Case Studies | April 15, 2012, 7:00am AEST
WOOLLAHRA Council’s Gap Park video surveillance system is one of the most engaging CCTV applications I’ve ever covered. A key element of Woollahra Council’s Gap Park Masterplan, the evolving system at Gap Park is designed to reduce self harm attempts at the iconic site through a multi-pronged approach incorporating fencing, onsite support for distressed individuals and intervention where possible.
Video surveillance serves 2 functions at Gap Park. If a distressed person is reported in the area by a member of the public, they can be sought by a control room working in conjunction with police response teams. Importantly when it comes to this upgrade, the system operates 24 hours and at night operators view the site using thermal cameras.
Before we get any further here it’s important to point out that this system is not a guarantee of safety, nor does it indicate NSW Police’s responsibility to reduce self harm attempts at Gap Park. Instead the system is the material expression of a large number of interest groups each seeking to mitigate the suffering of individuals and their families.
These interest groups include Woollahra Council, NSW Police, Lifeline Australia, Black Dog Institute, NSW Parks & Wildlife, the Federal Government and inevitably, the security companies involved in its design and installation including installer Kings Security, The Monitoring Centre team and Security Consultants International’s Daniel Paul.
Any system designed with the express purpose of saving more than 50 lives a year comes with a high level of emotion attached to it. All those involved in the system are engaged by the intensity of its purpose. Further, this is a complicated installation and the security companies involved do their work with little or no margin. Speaking of Gap Park consultant Dan Paul told me: “It’s a love job”. And I knew exactly what he meant.
The initial system deployed Pelco Spectra 4 PTZs, Bosch MIC 412 thermal cameras and Dallas Delta Help Points supported 24 hours a day by Lifeline Australia. Comms were handled by fibre in directional bores – a major early expense but excellent infrastructure moving forward. There are also customised poles and fittings throughout.
Stage 1 saw low-level LED lighting in some parts of the park, as well as Help Points with a hands-free speaker and built in microphone. These Help Points also have an ID giving location information and they incorporate vandal resistant aluminium housings and weather protection. The system is installed partly on a National Parks & Wildlife site and the head end of the system is located in an NP&WS building adjacent to Gap Park.
The way the system functions is that it can only be accessed remotely by an independent monitoring station and in the event an incident is reported to police by a member of the public, officers will contact the monitoring station seeking confirmation of the event, while simultaneously initiating a response.
This circuitous response mechanism reflects the requirements of the parties involved. While NSW Police responds to incidents and bears the brunt of effort on the ground, responsibility for monitoring a system like this one would be a significant burden on strained police manpower. As a result, police have chosen to remain at arm’s length from system operation.
Nevertheless, the central function of the CCTV system at Gap Park is that it saves police time when searching for distressed people – seconds can make the difference. At night thermal cameras ensure the monitoring station can try to help police establish exactly where a person is on the site if possible. Driving the Stage 2 upgrade was enhancement of the operational nature of the Gap Park surveillance system that was limited by Stage 1.
“Cameras at hot spots give a record of evidence and help in police investigations but they still have weaknesses. Police may race up to Gap Park but there’s a kilometre of rough bushland and they have no idea where to start looking. It can take several hours to search the cliff area.”
To find out more about the latest upgrades to the system I sat down with project consultant, Dan Paul who has been involved from the beginning. According to Paul, his involvement in the project began when Woollahra Council put its name down for a tender meeting he was conducting with Gosford Council. On a hunch Paul rang Woollahra Council and asked them if they were looking for something specific and after a number of discussions it became clear what was needed was a scope of works for Gap Park.
“At the time I went to council and said ‘we need to write a scope of works – what are you trying to achieve?’. And they said ‘we’re trying to reduce the number of suicides every year with government funding.’”
Paul says he was moved by the importance of the Gap Park system and he walked over the site after having a proposal quote accepted and came up with a comprehensive design incorporating analytics and 40 thermal cameras spread along a fence line.
“This first design was deemed too intrusive, so I came up with the idea of hot spots at the entrances so we could concentrate cameras at entries and key areas where people are in danger,” Paul explains.
“Cameras at hot spots give a record of evidence and help in police investigations but they still have weaknesses. Police may race up to Gap Park but there’s a kilometre of rough bushland and they have no idea where to start looking. It can take several hours to search the cliff area.
“As time went by I started to get a greater feel for Council’s problems. I spent 20 hours considering the terrain and seeing how people moved around the site before coming up with the new design,” Paul says.
After this design was accepted, Stage 1 of the project was then installed, including cameras and Help Points.
“What Stage 1 of the installation offered was the ability, if a member of the public called the police and informed them of an event, for officers to contact control room operators and ask them to fire up the surveillance system and find a person so as to allow police to intervene more quickly.
“It also gave a recording of events as well as the onsite ability to assist via telephone if the distressed person decided to use a Help Point,” Paul explains. “To a certain extent the system has been very good but it is reactive so it’s never going to proactively resist self harm at Gap Park – more was needed.”
With Stage 1 installed and functioning well, the interest groups involved with the Gap Park project were able to spend time assessing strengths and weaknesses of the system and carefully consider their next moves. These lessons were reflected in the next upgrade of the system which included more optical cameras as well as strategically placed cameras with the ability to deliver of thermal and optical imagery at the same time.
“To handle this task we are using the MOOG product which lets us view the terrain day or night in thermal or optical simultaneously and as you’ll see later on, this ability is highly advantageous,” Paul explains to me.
“We also wanted improved networking ability and system management and we turned to Geutebruck for this. We needed a system that could work effectively over an ADSL line and the Geutebruck GeViScope –HS/+ and GSCView software were ideal for the task. Operators can now view recordings to help them in their searches and our procedures were changed to facilitate this. Going with Geutebruck was a really good decision – it’s an excellent solution.
“At that time I was thinking of setting up 2 ground-based radars at 2 strategic locations at Gap Park to look at the surface of the terrain. With radar you can be very accurate – you can draw a line and as soon as an object moves into an area the system generates an alarm giving some forewarning.”
As Paul explains, establishing the location of people on the site with accuracy is vitally important.
“Experience has told us that when operators talk to police they say, ‘we can’t give you specific details of where a distressed person is in the area’ and police say ‘but we are trying to help you – so we need you to give us that detail’.
“This is a key challenge of the system and over the period of the last year as images have been retrieved by engineers at Kings Security for police investigations, the team has been able to get a very good idea of common areas and focal points,” Paul explains.
“To be honest what’s out there is mostly vapourware when it comes to video analytics. But what Geutebruck was offering with its IVA was intelligent motion detection which I knew had been working for years in prisons. I thought, I wonder if you can do motion detection with thermal cameras.”
Stage 2 progressed well with the swap-over to the Geutebruck-HS+ system being easy. This Geutebruck is styled as an omnibrid enterprise surveillance system thanks to multi-standard video compression including M-JPEG, H.264 multimedia, H.264 CCTV, Mpeg4CCTV, Mpeg4CCTV/MP. This flexibility allows the unit to support resolutions of CIF, D1, QCIF, HD and megapixel. There are 16 inputs, 4 HDD bays and 25/30 frames per second recording.
The key thing with unit was its image processing capability which is very comprehensive. You get basic activity detection, advanced activity detection, VMD, dual sensor, evaluation for objects missing based on parametrization, number plate recognition, and video content analysis for IP. It was the video motion detection that was perfect for Gap Park.
Also installed were additional cameras and this part of the installation was relatively straightforward as well. New cameras gave coverage of Help Points so as to better identify distressed persons, while MOOG thermal/optical cameras enhanced performance day and night.
“As well as enabling much greater capacity to remotely monitor and view images live and recorded, the Geutebruck system also provided us with the ability to undertake trialling of either system analytics and or potentially ground based radar for detection and monitoring of persons breaching designated fence lines in known areas of concern,” Paul explains.
“The current system allows us to record images across numerous cameras strategically situated for both day & night images, to remotely access the cameras live through the nominated and authorised monitoring centre and to review images for post incident investigations. Stage 2 allows us to bring closure to families and police investigating particular incidents as well as providing live feedback to authorities and lifeline to determine the whereabouts of persons of interest.”
At this point, Paul starts driving the system in real time using his GSCView client. The optics and the thermal images move together when the PTZs are driven giving awesome ability to see where people are.
“On a grey rainy day like this you can see the benefit of thermal,” he says. “You can’t see anything with the optical in this view but with the thermal you can see things you’d never pick up visually. In the night time the view is identical in terms of quality so if the police are walking around the site with a torch and the operators can direct them using this footage that’s excellent.”
I consider the scene. While it’s not a clear or crisp image you can tell there is a person in a given scene, you can see where they are and you can tell what they are doing. The distance the thermal cameras can detect humans is amazing.
Paul pans about and looks at people moving around the site, at the gun turrets, behind Jacobs ladder – at all times it’s possible to instantly be certain there are people moving even at extreme distances. I can see the value of the thermal cameras. You’d have no hope of seeing people with ordinary cameras or the naked eye at these distances.
“The cameras we have here are 2 CIF but we are trialling 4CIF thermal cameras elsewhere and they are unbelievable,” Paul says. “The best thermal cameras offer superb resolution and depth of field and will offer correspondingly excellent discernment in terms of detecting human presence in difficult areas like Gap Park.”
Next we look at the new cameras at hot spots. They clearly allow operators to identify people, as they pass through into the park.
“We can see the colour of hair and clothing – it allows us to ID individuals – that’s the key to these hot spots. If someone leaves a note saying they are coming here there are only a limited number of ways into the park and we can immediately see if they arrived and if they were alone,” Paul says.
While the Stage 2 works were designed to allow testing of analytics or ground radar Paul says he was delighted when the Geutebruck team told him they would throw in Geutebruck analytics for the trial.
With Stage 2 under our belts and Geutebruck’s analytics offer on the table about 3 months ago we said let’s trial these analytics to see whether they can help us to help police at Gap Park,” Paul says.
There were some challenges. The MOOG thermal cameras Paul wanted to use for this trial are not fixed cameras but PTZ units and analytics really needs a fixed camera and a locked field of view.
“To get around this we instructed the control room not to move these 3 MOOG cameras and we also set up a preset as a home position – just so if the cameras are moved, they’ll swing back ‘home’ ten minutes later and give us our analytics again.”
Of course, it’s one thing to talk about IVA in theory and something else entirely to implement it successfully in the field and before this installation Paul says he did not place a lot of faith in the technology.
“Up until this trial if a client asked me if analytics worked I would always say no, it does not,” Paul explains. “To be honest what’s out there is mostly vapourware when it comes to video analytics. But what Geutebruck was offering with its IVA was intelligent motion detection which I knew had been working for years in prisons. I thought, I wonder if you can do motion detection with thermal cameras. But given my doubts it seemed best to do some testing before I spoke to Council.”
Paul says the lads from Geutebruck brought their gear in and set it up. Because most analytics solutions are bolt-ons, the fact this was designed to work with Geutebruck product made things much easier – there was no struggle to get 2 disparate systems talking to one another. Paul has access to the system through a GSCView workstation client and the motion detection analytic was loaded onto this workstation.
Geutebruck’s analytics come in the form of its VMD – Video Motion Detection product, a 3D video motion detection solution that calculates at programmable time intervals (40 milliseconds to 10 seconds) a measurement for up to 128 VMD fields in each scene. The VMD analysis algorithm then compares these measurements against a baseline and calculates global and local changes. The VMD fields can also be set on the basis of direction of transit or on speed.
Neat features of the system include three dimensional analysis using automatic perspective selection and automatic switching of operational modes including day/night and working times. Using other test cameras mounted in his office to show me how, Paul starts creating motion detection barriers in camera views outside his window. He praises the integral aspect ratio of the software that allows it to recognise objects at great distance are larger than they appear. The process seems very simple to me. Motion detection areas are simply defined and saved – the same process that was applied to Gap Park’s cameras in the trial.
“We set up layers of detection in the trial at Gap Park,” Paul explains. “We can set it up so the software will trip an alert if this layer is breached, give you a warning there depending on intrusion depth and breach parameters and go into full alarm if the intrusion proceeds further. You can see as people go through the detection cells they change colour – that’s the Geutebruck analytic in the background.”
There are reference cells too, so the system can handle variation against a changing background which is important at a difficult site like Gap Park. Paul says that when the analytics were trialled the results were phenomenal at every level.
“As well as be able to view every intrusion over the barriers in the areas under surveillance, what we found is that there are patterns to intrusion into danger areas,” he explains.
“These are tourists – you can see people are standing at the fence taking photos. If they are standing there we don’t want an alarm. And if they lean over the fence for a better view as people often do, we don’t want to trip an alarm then either,” he explains.
“But if they are sitting in that yellow area there which is actually on the barrier and in breach of the fenceline for a significant period of time, then we do want to raise an alarm. And if they go from the yellow area which is the barrier to the red area which is the cliff top, we also want to trip an alarm. Of course, if something comes from the red to the yellow, we don’t want to trip an alarm. It’s probably a seabird or a cat.”
According to Paul, the team set the analytic up on several cameras in different areas of the park.
“We came back a couple of weeks later to view the footage and see what had been detected by the analytic,” he explains.
Now Dan shows me recording to indicate what they saw. It’s in the monochromatic coloration of a thermal camera and somehow this makes the events playing out before me even more stark. All this footage is captured late at night or in the early hours of the morning when no one else was around.
“What we found was that in most cases distressed people are on the cliff top for quite a while. They will climb the fence, lean on the danger side of the fence, and they may approach the cliff edge, then return to the relative safety of the fenceline in a process that takes half an hour or more. What this means is that in some cases there could be a significant window for early response.”
We view striking footage. From the left of the screen a figure appears and stands near the fence for a couple of minutes before climbing the fence and sitting on it at 2.51 am. At 3.05 the figure climbs all the way over the fence and into the danger area.
“It’s at this point an alarm can be raised – though in this case the system is not set up to alert operators in the monitoring station, nor are procedures in place to respond,” Paul explains.
At 3.07 the figure walks upright to the edge of the sheer cliff and then at 3.10 disappears out of camera view.
“Let me tell you when you’re sitting there watching the footage and something like this happens it’s a huge shock,” Paul says. “You just hope the person did not fall off the edge.”
According to Daniel, this footage proves perfect not just for early warning but for possible use in studies of self harm at similar locations. And this was brought home by what happened next. About 37 minutes after leaving the view of the camera and walking towards the cliff, the person comes back and sits down at the fenceline for a time, then gets up climbs back over the fence and leaves.
“It’s a long event that involves an individual standing, sitting, pondering, going over towards the cliff edge and being there for a long time before deciding to leave the area,” Paul says. “What was fascinating for us as part of the system’s role of reducing these events was that this was not a one-off incident but it was a pattern.”
We watch another segment of video.
“Here’s an incident at 10pm captured by the thermal cameras running VMD analytics. In this case the person climbs over the fence into the danger zone and sits down for an extended period in thought – 28 minutes – then climbs back over the fence and moves away. And there are other incidents that are all identical.”
According to Paul, this trial using thermal cameras supported by analytics shows that there’s even more potential for the system to help save lives and at the same time, there’s no need for expensive radar solutions.
“The thermal and analytics combination works way better than I ever thought it would,” he says. “The cameras provide video display and analytic detection simultaneously and the system can be set up to allow the associated and existing pan, tilt and zoom (PTZ) cameras to work in concert with the fixed view cameras to redirect the PTZ cameras to provide the operator a wider view, more accurate details and fast response without the need for the user to take control or intervene if they are not confident or unaware of the operations of the system.”
When we first reviewed this system, Woollahra Council’s Rod Ward explained to me that analytics could benefit the overall application by allowing video response to be based on pre-configured parameters like particular movements in particular areas at particular times.
“I always thought analytics could be beneficial and there has been some discussion about what could be done with analytics to make the system proactive,” he explained.
“Analytics could speed up this process – if the person was last seen in a red shirt, the computer could do the search and find the person virtually instantly. I also like the idea of analytics being able to identify the fact someone has been standing at a fence for 20 minutes and then raise an initial alert.”
Looking at the performance of the Geutebruck solution currently being trialled on the Gap Park system it’s immediately clear that Paul’s testing has proved Ward absolutely right. Analytics could be the perfect enhancement to a very capable system, an enhancement that would give police response valuable extra minutes.
“Based on these results, we are confident, with correct and strategic placement of a small number of additional fixed view thermal cameras, we could establish a permanent analytic fence line which would enable us to raise an alert to a suitably equipped respondent,” Paul says.
Listening to the tone of his voice, there’s no doubting the sense of urgency that is shared by all involved with this system and when you look at footage collected from the site it’s easy to see why. Here, in this unusual lateral application, is a CCTV system that makes a difference.
“Gap Park has become a love job for all of us involved with it – no one is making any money – we just want the system to save lives,” says Paul.
“And after the success of these trials, there’s motivation now to reconfigure the system and its procedures in a way that allows us to use analytics to try to intervene earlier if we possibly can.
“If we typically have half an hour and we know exactly where a person is then there may be an opportunity to respond with greater precision. This will allow police officers more time to try to talk people through their distress. And they do. Talking to someone who cares is often enough to change people’s minds.