Metrics & CISO Series

How well are digital forensic laboratories coping and performing, whether accredited to ISO17025 or not, in a Covid-19 world? Is Metrics relevant to digital forensics? 

In the first instance it largely depends whether there is a need for Qualitative and Quantitative (Q&Q) process in place that requires measurement to understand Capex, RoI, the performance of people and systems (for test and measurement), security (detection and prevention), information security management and so on. Where labs are tax payer funded then of course they should be scrutinised irrespective whether they are law enforcement or not. This isn't a criticism about public funded labs, but there does need to be a distinction made where private labs use capital expenditure to drive their operation to gain a return on investment. For both public/private large organisation Metrics shines a spotlight on the operational performance of these organisations (successes/failures).

In the second instance the simple is Yes but subject to if (?) the requirement exists of course. The question mark arises as to whether any implementation has taken place. Talking about it is one thing. Acting upon it is another. If the second question is considered first, it may reveal what impact there has been since Covid-19 and where to target resources.

Back in 2013 when the rumblings about ISO17025 were gathering pace a book came out titled (ISBN: 978-1-59749-742-8)  David Watson & Andrew Jones Digital Forensics Processing and Procedures Meeting the Requirements of ISO17020, ISO17025, ISO27001 and Best Practice Requirements Copyright 2013 Elsevier, Inc. This book shed light on the processes and procedures to run an accredited laboratory under ISO17025 plus associated dependency standards ISO17020 and ISO27001.  

There are other books, but as starting point Digital Forensics Processing and Procedures Meeting the Requirements of ISO17020, ISO17025, ISO27001 and Best Practice Requirements has multiple landing points in the book dealing with the need for Metrics. Furthermore, ISO27001 concerning information security is a very important standard, as is ISO9001 regarding quality assurance. The detail in this book extended further to provide to the broadest extent possible various ISO standards having application to accredited laboratories.

In addition to the above International standards there are national standards to be considered and Guidance from authorised bodies to oversee compliance. In the UK the Forensic Science Regulator (FSR) over sees the requirement for accredited laboratories. The latest FSR publication titled 'Codes of Practice and Conduct for forensic science providers and practitioners in the Criminal Justice System FSR-C-100 Issue 5' can be downloaded using the weblink below :

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/880708/Codes_of_Practice_and_Conduct_-_Issue_5.pdf 

The Codes of Practice and Conduct identify standards etc the FSR considers fundamental to be able to use to assess compliance. Moreover, the Codes come with dependency obligations and one notable one is 'Information Legal Obligations FSR-I-400 Issue 7'. There is contained in this document the express requirement for production of 'Metric' results as a legal obligation which FSR-I-400 takes its direction from European Union Directive 80/181/EEC that sets out obligations on Member States to implement legal requirements with regard to the use of units of measurement. The FSR makes express use of these legal obligations that create implied terms that Metrics equally form part of the Codes assessment for compliance. The latest FSR-I-400 can be downloaded using the weblink below :

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/795995/FSR_Legal_Obligations_-_Issue_7.pdf

It is the above matters arising from earlier released documents from FSR and various digital forensic books that set my project path for the last 15-months to see how the Professionals in the information security management, cybersecurity, risk assessment etc sectors identify the indicators to be sampled and measured for Metrics and how quality and accuracy is defined in these sectors. I have slowly been publishing my findings in a series of discussion papers and these can be downloaded using the weblinks below :

Metrics papers for cyber security & CISO.pdf Colourful, glossy, high-quality imaged research publications can look really good, but ultimately it is the depth of knowledge that has been gained from researching is what pays off. Why? How do you know your Metrics criteria is relevant to your organisation if you weren't sure what questions to address at the get-go?!

https://www.dropbox.com/s/kqsdo3dpsu2k03k/Metrics%20papers%20for%20cyber%20security%20%26%20CISO.pdf  

Importance of Metrics - The opening 'Foreward' in George Campbell's book "Measuring and Communicating Security’s Value A Compendium of Metrics for Enterprise Protection" (2015) [Elsevier - ISBN: 978-0-12-802841-4] is by Dave Komendat, Chief Security Officer The Boeing Company. Dave refers to why it is compelling how important Metrics are to a CSO.

https://www.dropbox.com/s/94ek2jxtrwwt3um/Importance%20of%20Metrics.pdf

Metrics, CPS & CISO - This is my third byte-size posting on Metrics. The pdf is only a scoping documents discussing in an understated conversational/readable way dealing with a subject-matter that is far more convoluted, complex and complicated.

https://www.dropbox.com/s/4u3c8lyn2k1gxx1/Metrics%2C%20CPS%20%26%20CISO.pdf

Metrics - Quality, Accuracy & more and CISO Part1

https://www.dropbox.com/s/t9sk46grg2p7xi9/Metrics%20-%20Quality%2C%20Accuracy%20%26%20more%20and%20CISO%20Part1.pdf

This discussion will be updated with further discussion papers to download.

Examination Techniques8: Simple Experiments2

Examination Techniques8: Simple Experiments2

Continuing the discussion to offer suggestions on ways to generate test methodologies in order that down the line it might be possible to create validation and verification processes, practices and procedures for mobile phone examination (device under test (DUT)). 

Assuming that an examiner is satisfied as to when s/he is using the appropriate tool that it will extract and harvest data from the make/model (DUT) under examination, there will still be the prior query what exactly is this tool communicating to the DUT? For instance, considering logical data as opposed to physical data, does the tool intended for use provide an "output log" that contains the communications (commands) sent to the DUT (e.g. APDU or AT+ etc etc) so that the examiner:

- can corroborate what is being instructed to the DUT when that tool is applied to it?
- comprehend are the responses (data) received from the DUT to be expected or are the data incomplete?
- Should the data be incomplete, is that because the commands are incorrect in their instructions which data are to be extracted or is it because the handset has not stored any further data other than that data returned in response to the command sent?

The objective of this simple experiment is to observe the content of any harvested logical data so that when dealing with physical (deleted) data recovery an examiner can start to build a template, from known logical data samplings, in order to apprehend some understanding of any deleted data that has been recovered as to what maybe there and what maybe missing.

Previous discussions relevant to this topic:
Examination Techniques6: Simple Experiments - http://trewmte.blogspot.co.uk/2012/03/examination-techniques6-simple.html

Examination Techniques5: Validation and Verification - http://trewmte.blogspot.co.uk/2012/03/examination-techniques5-validation-and.html

External links:
http://www.forensicfocus.com/Forums/viewtopic/t=8879/

Examination Techniques6: Simple Experiments

Examination Techniques6: Simple Experiments

Leaving validation to one side, when I am teaching mobile phone examination I get delegates on the course to try lower level tests, at first instance.


Try this experiment:

1) Conduct acquisition and harvesting of the handset's SMS text message logical data.

2) Produce a paper printout report of all those text messages (this will be one test guide).

3) Through the handset reading tool you are using display the text messages on the screen of your computer (this will be another test guide)

4) With the test handset switched ON view the text on the screen and take screen shots. The information in the screen shots should be as complete as that which can be viewed on the screen of the handset by the ordinary user (this will be yet another test guide).

The purpose of this experiment is, having cross-referenced all the three test guides, to see if they are, in the first instance, identical in every way? Moreover, can the tests be replicated by an examiner with the same system or another system?

Another simple experiment to consider:

To consider and, through trial and error, discover when would you apply a hash value?

Does your tool currently produce a hash displayed on the screen for each text message or are all of the saved text messages given a hash value?

Specifically, when your computer produces the output of data on to printed paper, is a hash value displayed (somewhere) and, if so, is the hash value the same as seen in the program on the computer screen or is it the hash value for the data that is actually printed on the paper (or would you need another value for that)?

With respect to the handset screen shots, would they need a hash value and would the hash value be created by the screen shot program for the images or the printed out data.

The purpose of this experiment is to identify exactly the relevance of hash values and to what they are being attributed, to what they technically prove (or who they exonerate), apart from having loads of hash values needing to be explained to a Court where the hash values do not exactly corroborate each other, but different things.

Now re-run the experiments with two different handset readers.

Previous discussions about some issues associated with validation and verification:

http://www.trewmte.blogspot.co.uk/2012/03/examination-techniques5-validation-and.html

http://trewmte.blogspot.co.uk/2010/06/evidence-and-admissibility-part-1.html

http://trewmte.blogspot.co.uk/2008/12/nokia-6233-clock.html

http://trewmte.blogspot.co.uk/2008/11/mobile-phones-and-fringe-coverage.html

Examination Techniques5: Validation and Verification

Examination Techniques5: Validation and Verification
The constant and never ending challenge to ensure examination tools meets the requirements when used with or applied to the DUT (device under test) naturally generates constantly evolving polices, practices and procedures. The 'digitally-evolving' and 'technology-fast development' age has brought with it an inability to keep policies, practices and procedures up-to-date. That includes the tools and techniques that maybe used or applied.

Examiners may find it helpful when dealing with Validation and Verification:

- To validate is to assess doing the right things,
- To verify is to evaluate doing things right.

As always, Wikipedia has interesting articles and prompts that can help start researching points:

http://en.wikipedia.org/wiki/Verification_and_validation

However, a word of caution and a useful philosophical approach to remember, born out of having previously had experience and worked in QA, factory assessment/evaluation pre-approval and technology assessment. When considering Validation and Verification they hold the same unlying warning as that attributed to Galileo, who said "the Bible shows the way to go to heaven, not the way the heavens go". Thus agreeing that validation is being performed does not necessarily mean verification will automatically support that claim.

Exploding Cell Phone Battery

Exploding Cell Phone Battery

.
A reminder for examiners to take care when examining mobile phones that require recharging the battery before examining the device. The horror story of a man's who had his neck artery burst by the exploding mobile phone in his breast pocket and died. Prior to putting it in his pocket the man had recharged the battery. Production line (bang it on, bang it out) mobile telephone examinations are potentially at high risk and a risk assessment procedure may need to be put in place to avoid loss of fingers or limbs or disfigurement. Insurance companies may need to be made aware of this potential high risk.
.
http://www.telegraph.co.uk/news/worldnews/asia/china/4452360/Exploding-mobile-phone-kills-man-in-China.html
.
Last year Vinny Parmar produced a document (see link below) about counterfeit mobile telephones and it is by no means certain whether these devices are using batteries that may explode.
.
http://trewmte.blogspot.com/2008/11/counterfeit-mobile-phones.html
.
Moreover, some people are apt to placing their mobile telephone in more unusual places about their person. Whether the person is smuggling or something else the exploding battery provides a stark reminder that they may put their life at risk.
.
http://trewmte.blogspot.com/2006/11/whats-this.html
.
Thanks to Richard Putnam for sending to me the Telegraph newspaper online link.

.

UPDATE: A further Telegraph online article has been written regarding the above exploding battery. Apparently, this second news article suggests it was a bullet from a home made gun that had cause the accident. However both Telegraph online articles are not referenced to each other so it is only an assumption that they both may be referring to the same incident.

http://www.telegraph.co.uk/news/worldnews/asia/china/4522915/Man-killed-by-exploding-phone-shot-himself.html