Week 3 update!

We are excited to share our progress update for week 3! Our team met with our problem sponsor, military and defense agencies, academics, and tech firms this week as part of our discovery process. We have also spent a fair amount consolidating this information for our presentation this week!

Brainstorming Session #1

Brainstorming Session #2

 

Problem Sponsor: JIDO 

Interview #1 with problem sponsor: Brennon D., Ben C., Mike T., and Angela L., JIDO FR project team, Washington D.C. metro area

Main takeaways:

• Collaboration with Department of Homeland Security
• Data transmission and linkage will be a bottleneck because of DoD
• The best way to do this is to have forward servers/ local instances to process the information
• Needs to think about human machine interface

Other Notes:

• Currently BAT/HIIDE is slow and invasive. Creates agony and tension within local community
• Defense Forensics Biometrics Agency (DFBA) is the overwatch that handles information
• Biometrics Enable Capability (BEC) is the current upgrade program for DFBA’s information processing system. Takes long time.
• TSA wants to improve complacency and safety
• Think about collection on US personnel and US soil

---

Military & Defense: 

Interview #2 with Mr. TOR, Office of the Chief of Engineers, Pentagon

Main takeaways: 

• Make it happen AND make it stick

• It’s one thing to develop a solution for a challenge.  It’s another to being able to have it endure as part of the defense apparatus.

• There is a lot of reluctance on the part of the Branches of Service to adopt the next “hot item” or “cool gadget” because what the innovator/developers don’t take into account is the role of doctrine, policy, acquisition, procurement, and maintenance that goes into having a piece of tech as part of the functional unit on the ground.

Other Notes:

• Walked through the evolution of the Counter-IED fight from 2002 onwards.
• Started with General Cody setting up the Army Counter-IED Task Force and then General Votel of SOCOM (in 2003, he was BG Votel) developed the question of “Defeat the Device vs. Defeat the Network?.”
• That question – Device vs. Network – led to the continued evolution and development of the entity that would become JIDO today.
• What eventually became the Joint Improvised Explosive Device Defeat Organization (JIEDDO) was the union of two entities or rather parts of two organizations – The Rapid Equipping Force and the elements of a Joint Task Force.
• This was Deputy Secretary of Defense Gordon England’s legacy – an Joint organization that could access across the services to develop counter-IED measures with the urgency and expedited nature of a rapid acquisition cycle.
• This led to a lot of reluctance on the Services side because for every gadget that JIEDDO, it was difficult to maintain in the inventory because a capability need had not been codified for that widget.
• Have to remember the JCIDS process as this will make “things stick” in the Department of Defense.
• Like it or not, it is a reality of the DoD
• Something like this:
  • “What does the future look like?” (regardless of the next war, next piece of tech, next activity a soldier has to do).
  • CBA – Capabilities Based Assessment – identifies a requirement
  • ICD – Initial Capability Document – the requirement you want to develop
  • CDD – Capability Development Document – starts to build the parameters of the requirements
  • CPD – Capability Production Document – how to build the widget that will do the capability – i.e. the initial specs.
  • KSA – Key System Attributes
  • KPP – Key Performance Parameters
• Like it or Hate it, JCIDS is what helps to make innovations in the DoD stick beyond the current budgeting cycle.  
• If an organization within DoD discounts that process or doesn’t take it into account, they do so at their own risk.
• Recommended that we connect with the Program Manager at the Engineer Research & Development Center in Vicksburg, MS as well as his connection at Department of Homeland Security to continue the discussion of capability needs when reducing threats in an area.

Interview #3 with Captain MS, USACE District within Mississippi River Valley, served in OEF as Platoon Leader (Frontline Supervisor).

Main takeaways: 

• If something is “wearable tech”, think about how heavy it will become in 12 minutes vs. 12 hours and how effective someone is going to be because of that.

• In order to more effectively action the biometric data, it has to have a faster processing time – real time uploaded and analyzed.

• There needs to be a means to upload the data either remotely or connected to the network somehow.

Other Notes:

• Served as a Platoon Leader in Paktika, Wardak, and Logar Provinces of Afghanistan in 2012.
• Part of his mission set was conducting foot patrols and establishing check-points on the ground in order to control the movement of people and vehicles in the area.
• One of the tasks for these check-points was to “BAT/HIIDE Everybody”.  Higher Command simply tasked the guys on the ground to collect images and biometric data of people who came through the checkpoints – as part of the effort to build out the database.
• MS and his guys were given BOLO (Be On Lookout) lists of persons of interest to detain and secure should they come through the checkpoint.
• Pains of the BAT/HIIDE System
  • “Camera Optics Sucked”
  • Heavy – try holding it up and steady to capture an image for longer than 5 seconds.
  • The finger print scanner would get “gunked up” – reminder: the population is predominately rural in Afghanistan, farmers and laborers – dirty hands.
  • They’d be able to get through maybe 5 people in an hour.
  • Once the data was captured, it would sit on the device and have to be physically transported to an intel section at an operations center to be downloaded and analyzed.
  • Once they had a person of interest (flagged on a BOLO list) travel through their checkpoint and only get notified of it 3 weeks later once the data had been analyzed – much to their irritation and frustration.
• Improvements or Gains thought of:
  • Better Optics and Camera System
  • Upgrading the hardware so it is rugged, durable, and yet light enough to handle.
  • Needs to be able to speed the transmission of data
  • Real Time development and analysis is needed if you’re going to close the loop faster.
  • How network capable is it?
  • If you get a positive hit, how is that getting transmitted to the person on the ground?
• Food for thought:
  • If it is wearable tech, prefer not much heavier than a pair of glasses or equal.
  • Think about how long you are wearing it – on a 12 hour shift, things get heavy.
  • Centered and Balanced vs. Not Heavy
  • Ruggedized – Think about the “simplest” person in your security detail.  What happens if you drop it?
• Communications in the system:
  • Through the device?
  • Through the tech?
  • Radio?
  • Tactical Satellite Phone?
• Think about your routine traffic stop that a police officer does.  The policeman inputs your license plate / license number / personal ID into his or her system, and the system processes it to find if there is anything outstanding – warrants, traffic violations, etc. The system notifies that policeman if there is anything outstanding and then the officer takes action.
• End Result of Interview: Beneficiary Archtype identified – Frontline Supervisor.

Interview #4 with Patricia Wolfhope, Office for Public Safety Research, First Responders Group, DHS S&T Program Manage, Department of Homeland Security

Main takeaways:

• Recommends contacting DOJ and police to start interviewing them about the body-worn cameras and how this applies to our challenge w/ AR + FR technology.
  • How does the data come into the system? How do you do a match? Where does it match? How do you alert an officer on what to do? Questions to consider asking them include:
• Protocols that have to be followed differ for facial recognition technology transfer – each agency has a different approach.

Other notes:

• Works for undersecretary of DHS in the Science and Technology. The charer is to bring forth R&D into operations, secret service, TSA, and USTIS.
• Her department informs operational end-users what might work
• Worked on the Face in Video Evaluation (FIVE) report, which will be coming out in February 2017. You can read more about this here.
• Provided helpful information on algorithms.
• FR can be achieved through cooperative (i.e. photograph at the DMV or mug shot where the person is complying with the camera) vs. noncooperative subjects i.e. when people are moving). It is easier to do a 1-to-1 match than a 1-to-many match.
• Suggests looking into end-users such as TSA and policeman — DOJ has been doing a lot of work on body-worn cameras. Other persons to consider include ICE, Secret Service, and emergency response.
  • Right now the body cameras are being used as evidence, not augmented.
• There are lots of policy/legal questions raised — how long do we keep the video on file?
• Database systems:
  • Smaller database more effective for faster transmission.
  • DHS doesn’t allow for wireless or bluetooth capabilities at this time (air transmission).
  • If you have to send pictures wirelessly, recommends just sending the photo rather than the entire video stream — makes this more efficient.
• Recommendations for further research/contacts: TASER company has body-worn technologies; also will connect us with someone overseeing body-worn camera research at National Institute of Justice.

Interview #5 with Mr. PA, Staff Sergeant in US Army Reserve, Intelligence Analyst with BAE Systems

Teuta and Dainis conducting phone interviews.

Main takeaways: 

• Collection of Biometric Data is the easy part.  

• Databases of biometric data are so large because of the sheer amount of information – personally identifiable information and information.
• In order to have telemetry and connect to a database, it isn’t enough to connect the system to a digital database.  You have to ensure the policies in place and the people who are cleared for access – which is a whole other process and friction point.

Other Notes:

• Down-side of the BAT/HIIDE biometric collections system:
• Don’t have direct linkage to a database that ultimately receives the information.

• Scanning database is part of the process to analyze for persons of interest.
• The database that is downloaded on the device is limited by the latest update to the system – time sensitive.

• When it comes to databases and biometric – the amount of data is massive.  
• Databases don’t speak with each other.  DHS – Immigration has their own – DoD – CIA – FBI – Border Patrol – Law Enforcement entities across the country.
• Lack of sharing is a big thing – Telemetry will help but that brings up policy.
• Work Flow:
• Collector is driven by the Analyst who is driven by the Commander, who then is driven by the information by the collector and the analyst.
  • The pushing out of data as quickly as possible.
• Pain:  SO Many Databases, there is no one universal data base.
  • If a database is within an agency, what’s the linkage? – I.E. who owns it and do you have permissions / clearance to get on the database?
  • It’s not like I can get a result from just one database
  • Policy – usually, it’s a process to find out who has administrative rights over the database.
  • “hey you this is me, I need information on your database, can I use it?”
  • Personally Identifying Information is stored on various databases.
  • Every 6-8 weeks, constantly refreshing permissions – Need to Know clearance
• Gain:
  • Million dollar question – how do you alleviate that pain?
  • Have to ensure the policies in place and the people who are cleared for this.
• Wearable tech:
  • I’d be willing to field test that.
  • Glasses make sense – have the ability to scan and capture data.
  • Body Cam and/or Go-Pro camera.
  • The collections is easy ; it’s the analysis part that is difficult.
  • Streamline it
  • Training the personnel using the gear – the bias of instinct.
  • “My background is in psychology” – you see what you want to see.
  • The objective camera lens can’t be affected by personal bias, personal attitudes, and prejudices.

---

Tech companies: 

Interview #6 with Matt Worsham, Amazon Web Services (AWS) 

Teuta meeting with Amazon Web Services.

Main takeaways:

• Do not conflate Databases with Data Transmission
• Hybrid models offer ways for the government to keep certain functions in their premises while obtaining others from the commercial tech company.
• The problem might not even be technical, rather a policy issue.

Interview #6 with N.S. the Senior Manager of Region Build Engineering  and N.R. software engineer at Amazon Web Services (AWS).

Main takeaways:

• Limited bandwidth is an issue if facial recognition tech is based on the ground
• AK uses snapshots to capture data not live video
• Data transmission a potential bottleneck
• Range depends on the quality of the picture taken (can give results of up to 4069 faces at once )

Other notes:

• If they use a local database or “link-back” option they could potentially cache locally for example “local terrorist list” but the accuracy may not be the same as list is limited and not updated real time
• if the “link-back” is done through the cloud, or data is on the cloud with a collection of more than 100,000 faces–the result is fast =100 milliseconds. Also, the cloud service is updated constantly and is a managed service. Security (encryption) of data not an issue  because they come with HTTPS socket.
• The software can set confidence interval and threshold, so that it may return the results of multiple possible matches
• Allows to link multiple pictures to a single profile, so that detection/identification will be more accurate.

Interview #8 with Director at Evolv Technologies

Main takeaways:

• Based on experience, recommends thinking about detecting other threats with AR + FR
• Should be focusing on accuracy – how do you detect false alarms/noise?

Other notes:

• A monolithic approach to security creates problems — security has to be massively distributed.
• There are soft targets around the world and high volume physical security in an unstructured environment.
• The Evolv Technologies company is made up of former engineers and policy makers that have experience building multi-model data algorithms that allow for the end-user to detect threats in a red light/green light action.
• Evolv currently offers a real-time threat platform that combines bunch of sensors filling out data …multiple data based on needs — artificial Intelligence and human judgment incorporated.
• In some places, build “portals” portals where you can walk through without divesting (can keep your keys and wallets on you) and it will determine whether you will have firearms
• Any sort of aerial/scanning view has problems with false detections. There are not just one but 20 unsolved problems in this.
• 3 types of sensors available:
  • 1.) military wave – big refrigerator-like machine – human skin acts like a mirror so you can detect objects between the person and scan. captures video frame
  • 2.) magnetic field – capturing metal
  • 3.) camera – electro-optical camera
• If you don’t have sufficient pixels across the eye on a ‘capture’ you cannot perform FR on this. Cameras combined with the subject’s distance from across the camera are how pixels are generated. Ideally you want to have uncompressed images, almost not the same. With a machine vision camera, it is possible to get uncompressed images from it regardless of light sensitivity, but this is usually not the case when hooking into an existing system.
• What gets you into trouble is when you try to stream video. Can be difficult to process.
• Typical threat detection softwares usually rank in cost of $150-250k but Evolv is making this affordable by offering technology for less than a third of the cost.
• Two ways for the end-user to detect threats: person on the ground located next to the sensors (next to the outer layer) or it can be a virtual outer layer — there is a line that divides sterile/non-sterile system
• Will follow up with other recommendations.

Interview #9 with Dr. Kirkley, WisdomTools Inc.

This slideshow requires JavaScript.

Main takeaways: 

• Gather and look for specific examples of what real situation would necessitate this technology to be employed (who is the end user?)
• There is sometimes resistance to change among departments for implementing such technology. Note that human and organizational behavior could prove a hindrance in implementing this technology and improving efficiency

Other notes:

• Company has worked on AR implementation with work on the following projects – more information here.
  • U.S. Army REDCOM project: Embedded training and performance support using AR.
  • U.S. Coast Guard project: Safer marine navigation made possible with vAtoN
• At the time, AR was used with binoculars so the end-user didn’t have to wear the technology at all times.
• Provided examples of former uses of AR tech (keep in mind that this was years ago)
• Recommendatins of people to reach out to:
  • NREL Research Lab – Battlefield Augmented Reality Systems
  • ARCortex Inc.

Interview #10 with Zhuqing Ding, Center for New Designs in Learning & Scholarship (CNDLS), Georgetown University

Main takeaways:

• How Google Hololens is able to work with precision and adjusting to any room environment

Testing out Hololens #1

Testing out Hololens #2

Other notes:

• Interested to follow our progress throughout the semester.
• Recommendations for further research/contacts: Interview Google Glass about lessons learned, new products

---

Academics: 

Interview #11 with Satya Venneti, Principal Investigator, Biometrics Software Engineering Institute, Carnegie Mellon

• Gave examples that clarified what counts as augmented reality and explained the utility of combining augmented reality with capabilities such as facial recognition software
• Validated our hypothesis that improving several components of a biometric system such as processing power, telemetry, and camera accuracy may improve FR AR capabilities. However, she also pointed out that improving workflow in military and security situations may be the best way forward considering the barriers to improving these components.
• Explained the potential value of incorporating wearable devices as a platform for capabilities combining AR and FR (these devices provide their own processing power, memory, high quality cameras etc.) In addition, users can multi-task while leveraging FR and AR capabilities. Supported hypothesis that FR AR combined are an asset for end-users.
• Emphasized the difficulty of ensuring the security of communication links in all types of engagements, urged us to focus more on ways to avoid data transmission problems.

Interview #12 with Ramy Guirguis, Lecturer in Technology Managements, School of Continuing Studies, Georgetown University

• Outlined more components of biometric systems and how they could be potentially improved (Capture- enhancement of image quality- matching algorithm…). Explained that accuracy is a big challenge and increasing accuracy increases processing time.
• Delineated the different types of functions that fall in the category of facial recognition and the unique challenges belonging to each function. This was useful because it made us realize that we need to work with our sponsor to understand which type of facial recognition we should focus on.
• Understanding the importance of accuracy and the trade offs of improving accuracy are major takeaways for our team. Our hypothesis focuses on improving a system but improving the components to maximize speed may be counter-productive. We need to understand how to maximize accuracy without sacrificing time. Understanding workflow may allow us to determine ways to balance the need for speed and accuracy. If we identify when, where, and how an end-user applies this capability, it can clarify the minimum requirements for data transmission, memory, image quality etc. In this way we can determine how to balance these factors.

Interview #13 with Professor Evan Barba, Assistant Professor of Communication Culture and Technology Program and Co-Director, Technology Design Studio at Georgetown University.

Main takeways:

• When approaching the data question, make sure to ask “where is the data and what format is the data in?” The problem regarding the data transfer is more likely one that focuses on units of the government/military not talking to one another and sharing information.
• Key question to ask is what sort of imprint are you leaving on this world when developing and implementing this technology – how do we verify accuracy in this technology? What are people’s rights?

Other Notes:

• Has a PhD in AR technology.
• The issue with facial recognition is that we need to know a full model of the person. The ‘unstructured area’ would have to be under surveillance in multiple positions to be able to identify multiple positions of the face.
  • Some challenges that arise: glasses and bears can pose problems with ID; lighting conditions may affect the way that computers read face.
• AR is a simple technology. The technology is there and has been for years. The interesting questions arise when you mix AR technology with more complex systems at difference scales while looking for longer term experiences and dramatic productions.
• Can the accuracy be translated from start-to-finish in 5 seconds? Not likely given the amount of data required to be processed.
• A lot of the technology depends on efficient algorithms.
• Recommendations for further research/contacts:
  • Look up Augmented Reality Standard Language