Metrics & CISO Series

How well are digital forensic laboratories coping and performing, whether accredited to ISO17025 or not, in a Covid-19 world? Is Metrics relevant to digital forensics? 

In the first instance it largely depends whether there is a need for Qualitative and Quantitative (Q&Q) process in place that requires measurement to understand Capex, RoI, the performance of people and systems (for test and measurement), security (detection and prevention), information security management and so on. Where labs are tax payer funded then of course they should be scrutinised irrespective whether they are law enforcement or not. This isn't a criticism about public funded labs, but there does need to be a distinction made where private labs use capital expenditure to drive their operation to gain a return on investment. For both public/private large organisation Metrics shines a spotlight on the operational performance of these organisations (successes/failures).

In the second instance the simple is Yes but subject to if (?) the requirement exists of course. The question mark arises as to whether any implementation has taken place. Talking about it is one thing. Acting upon it is another. If the second question is considered first, it may reveal what impact there has been since Covid-19 and where to target resources.

Back in 2013 when the rumblings about ISO17025 were gathering pace a book came out titled (ISBN: 978-1-59749-742-8)  David Watson & Andrew Jones Digital Forensics Processing and Procedures Meeting the Requirements of ISO17020, ISO17025, ISO27001 and Best Practice Requirements Copyright 2013 Elsevier, Inc. This book shed light on the processes and procedures to run an accredited laboratory under ISO17025 plus associated dependency standards ISO17020 and ISO27001.  

There are other books, but as starting point Digital Forensics Processing and Procedures Meeting the Requirements of ISO17020, ISO17025, ISO27001 and Best Practice Requirements has multiple landing points in the book dealing with the need for Metrics. Furthermore, ISO27001 concerning information security is a very important standard, as is ISO9001 regarding quality assurance. The detail in this book extended further to provide to the broadest extent possible various ISO standards having application to accredited laboratories.

In addition to the above International standards there are national standards to be considered and Guidance from authorised bodies to oversee compliance. In the UK the Forensic Science Regulator (FSR) over sees the requirement for accredited laboratories. The latest FSR publication titled 'Codes of Practice and Conduct for forensic science providers and practitioners in the Criminal Justice System FSR-C-100 Issue 5' can be downloaded using the weblink below :

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/880708/Codes_of_Practice_and_Conduct_-_Issue_5.pdf 

The Codes of Practice and Conduct identify standards etc the FSR considers fundamental to be able to use to assess compliance. Moreover, the Codes come with dependency obligations and one notable one is 'Information Legal Obligations FSR-I-400 Issue 7'. There is contained in this document the express requirement for production of 'Metric' results as a legal obligation which FSR-I-400 takes its direction from European Union Directive 80/181/EEC that sets out obligations on Member States to implement legal requirements with regard to the use of units of measurement. The FSR makes express use of these legal obligations that create implied terms that Metrics equally form part of the Codes assessment for compliance. The latest FSR-I-400 can be downloaded using the weblink below :

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/795995/FSR_Legal_Obligations_-_Issue_7.pdf

It is the above matters arising from earlier released documents from FSR and various digital forensic books that set my project path for the last 15-months to see how the Professionals in the information security management, cybersecurity, risk assessment etc sectors identify the indicators to be sampled and measured for Metrics and how quality and accuracy is defined in these sectors. I have slowly been publishing my findings in a series of discussion papers and these can be downloaded using the weblinks below :

Metrics papers for cyber security & CISO.pdf Colourful, glossy, high-quality imaged research publications can look really good, but ultimately it is the depth of knowledge that has been gained from researching is what pays off. Why? How do you know your Metrics criteria is relevant to your organisation if you weren't sure what questions to address at the get-go?!

https://www.dropbox.com/s/kqsdo3dpsu2k03k/Metrics%20papers%20for%20cyber%20security%20%26%20CISO.pdf  

Importance of Metrics - The opening 'Foreward' in George Campbell's book "Measuring and Communicating Security’s Value A Compendium of Metrics for Enterprise Protection" (2015) [Elsevier - ISBN: 978-0-12-802841-4] is by Dave Komendat, Chief Security Officer The Boeing Company. Dave refers to why it is compelling how important Metrics are to a CSO.

https://www.dropbox.com/s/94ek2jxtrwwt3um/Importance%20of%20Metrics.pdf

Metrics, CPS & CISO - This is my third byte-size posting on Metrics. The pdf is only a scoping documents discussing in an understated conversational/readable way dealing with a subject-matter that is far more convoluted, complex and complicated.

https://www.dropbox.com/s/4u3c8lyn2k1gxx1/Metrics%2C%20CPS%20%26%20CISO.pdf

Metrics - Quality, Accuracy & more and CISO Part1

https://www.dropbox.com/s/t9sk46grg2p7xi9/Metrics%20-%20Quality%2C%20Accuracy%20%26%20more%20and%20CISO%20Part1.pdf

This discussion will be updated with further discussion papers to download.

Not Comfortable Fit for Digital Forensics - ISO17025

Within the digital forensics arena there is discomfort amongst labs, academia, businesses and practitioners that ISO/IEC 17025 'General requirements for the competence of testing and calibration laboratories' is not a comfortable fit for digital forensics. Very few digital forensics laboratories and businesses have been accredited so far. To get an understanding of concerns obtained from a pretty good base-data of opinion from replies to UK ISO 17025 Digital Forensics Survey 4/24/2017 created by Professor Peter Sommer, the results have been published and are available here http://goo.gl/KP0HOn .

Not to second guess the Forensic Science Regulator (FSR) there is , of course, the October 2017 deadline looming and the outcomes of that deadline might impact on the way forward. However, I regularly keep an eye on Lab Accreditation and Best Practice Guides (as you can see from some of the pdf tabs open in the above screen shot) in context with digital forensics in order to note the changing approach to digital forensics. The new breeze appears to suggest digital forensics blowing towards ISO standards e.g.

ISO/IEC 27042: 2015. Information Technology - Security Techniques - Guidelines for the Analysis and Interpretation of Digital Evidence.

ISO/IEC 27037: 2012. Information Technology - Security Techniques - Guidelines for Identification, Collection, Acquisition and Preservation of Digital Evidence.

Currently, but this may change, these standards are not substitutes for accreditation. That does not mean though digital forensics may not branch off and have its own unique accreditation and standards. It may well be the British Standards Institute (BSI) may need to produce an equivalent standard for the UK based upon an example of the old BS5750 approach. BS5750 and ISO9000 do enable the UK Government's requirement to be met for "inclusion" of single-person organisations and SMEs to play apart in the economy and not be excluded from it due to globalism or restraint of trade practices or over-burdensome control measures.

Previously, I drew attention to how in the US, Karin Athanas, Program Manager at the American Association for Laboratory Accreditation (A2LA), produced an article titled "Accreditation for the One-Person Organization - The smallest laboratories can teach us the biggest lessons.". This article defined that smaller business entities could achieve accreditation to ISO/IEC17025: http://trewmte.blogspot.co.uk/2016/10/isoiec-1702517020-one-person.html

The UK ISO 17025 Digital Forensics Survey 4/24/2017 isn't the first time attention has been drawn to ISO/IEC17025 that it should works for all, not the few. If the latter accreditation doesn't work then maybe another route will need to be found.

ISO/IEC 17025/17020 - One-Person Organisation

Having just finished part two of the work study into QA and Laboratory Accreditation MTEB UK SEMINARS 2016 II v03- QA Lab Accreditation.pdf ( http://trewmte.blogspot.co.uk/2016/10/qa-and-laboratory-accreditation.html ) I came across this cracking article by Karin Athanas, Program Manager at the American Association for Laboratory Accreditation (A2LA) titled "Accreditation for the One-Person Organization - The smallest laboratories can teach us the biggest lessons." ( http://forensicfoundations.com/resources/Documents_CLR/2016_10_3_SUMMER.pdf )

Basically Karin's article helps us understand that one, two or three person/s organisation/s should not be put off but can and should apply for ISO/IEC 17020 and 17025 as the requirements are not insurmountable, particularly when it comes to allocation to whom the quality manager's role, audits etc. will be allocated and deemed to be responsible. I also read this to mean that ABs might need to widen their scope to appreciate many roles in a accredited system can be held by one person.

Karin's article is a recommended read.

QA and Laboratory Accreditation

MTEB UK SEMINARS 2016 II v03- QA Lab Accreditation.pdf
Read more - updated link http://tinyurl.com/zr2oqyp

QA and Laboratory Accreditation. Previously, lab criteria applied to mobile phone forensic testing was randomly applied:

- various industry standards
- public and private approach to best practice
- guidelines/training courses
- Some certified ISO9001, some sought UKAS accreditation.

The introduction of a UK Forensic Science Regulator (FSR), there are now mandated ‘Codes of Practice and Conduct’, standards and accreditation applicable to mobile phone forensic evidence:

- ISO/IEC 17025 e.g. requirements for the competence to carry out tests and calibrations...
- ISO/IEC 17020 e.g. scene of crime and in the field activity
- UKAS Accreditation

The FSR’s strategy moves the goalposts away from simply applying industry best practice and random approaches to a common purpose – provision of forensic science across the criminal justice system is "subject to an appropriate regime of scientific quality standards"

That common purpose approach has been developing for approximately 6 years but only really in the last several years ISO/IEC 17025:2005 has  started to make its mark and the first accreditation to requirements of Forensic Science Regulator’s ‘Codes of Practice and Conduct’ was  2014.

- still early days for the public and private sectors
- very small number of organisations accredited for mobile phone forensic evidence
- it could be said we are all pioneers to new endeavours
- common purpose does not dilute ‘speciality’ distinguishing one organisation from another
- FSR deadlines for public sector forensic science overall 2017-2020
- e.g. Law enforcement mobile phone forensic test laboratory accreditation by Oct 2017
- Lead times of 18-months to implement suggests 2017 deadline could be missed
- ‘devil in the detail’ causing much more work than at first thought

There is increased demand for practical solutions and helpful insights that may assist prepare for accreditation.

FSR positioning for statutory powers

You'll need to be quick to catch this one. 

Today, 3rd January 2014, the UK Government's consultation period ends regarding seeking views on whether new powers should be given to the Forensic Science Regulator.

Decisions are to be made which will impact on mobile, digital and computer evidential collection, forensic examination, acquiring evidence etc and you will need to decide where you or your organisation fit with the whole scheme of things.

https://www.gov.uk/government/consultations/new-statutory-powers-for-the-forensic-science-regulator#!

There is most likely to have been a wide range of responses from different industries and professions. One such response is the Forensic and Policing Services Association (FAPSA) and it is interesting to note that amongst its proposals they believe "high standard" is relevant. I take no issue with their stance and I am not suggesting for one moment this organisation hasn't always advocated such a stance. It is not always the case and some responses from other quarters, in the past, have suggested advocating "high standards" was to take an "elitist" attitude because 'some work to high standards and some work to low standards', which in my view was a strange stance to take regarding an individual's competency working in forensics and not in the interests of skillsets and standards to underpin forensic science.  FAPSA's stance communicates a useful message for those wanting to be or are involved with forensics science.

http://www.fapsa.org.uk/wp-content/uploads/2014/01/Forensic-regultaor-response-FAPSA.pdf

Another response to the consultation from Peter Sommer can be read here (see link below) and highlights some useful disparities between envisaged outcomes vis-a-vis the costs to implement them/the appropriate payment for such work:

http://pmsommer.com/HO%20FSR%20Consultation_sommer.rtf

What is clear is the need for those who never have to spend out of their own pockets should not set or allow vertical and horizontal market costs to rise by holding individuals or companies to ransom. This is quite important as there is too much reliance on the 'machine will do the thinking for me' and if my evidence and opinion is wrong then the human equation is not at fault. Knowledge, skills and experience are paramount to the work of forensic science. It is accepted devices are needed for acquisition but understanding how the device communicates with the target "thing" under examination (commonly we call this a DUT - device under test) and exactly what is being communicated to "action a command and receive a response" should be the prime facie case.

Useful examples of commands and responses can be found during the examination of mobile/smart phones. here are a few standards, but be mindful as standards extend to the air-interface (thus relevant to lawful interception etc) and mobile networks interfaces, too. 

Mobile/Smart Phones
GSM/3GPP TS 07.07 AT Command set for GSM Mobile Equipment (ME)
http://www.3gpp.org/ftp/Specs/archive/07_series/07.07/0707-780.zip

3GPP TS 27.007 AT command set for User Equipment (UE)
http://www.3gpp.org/ftp/Specs/archive/27_series/27.007/27007-c30.zip


Smart Card ICC/UICC - SIM/USIM
3GPP TS 11.11 Specification of the Subscriber Identity Module - Mobile Equipment (SIM-ME) Interface
http://www.3gpp.org/ftp/Specs/archive/11_series/11.11/1111-8e0.zip

3GPP TS 51.011 Specification of the Subscriber Identity Module - Mobile Equipment (SIM-ME) interface
http://www.3gpp.org/ftp/Specs/archive/51_series/51.011/51011-500.zip

3GPP TS 31.101 UICC-terminal interface; Physical and logical characteristics
http://www.3gpp.org/ftp/Specs/archive/31_series/31.101/31101-b00.zip

3GPP TS 31.102 Characteristics of the Universal Subscriber Identity Module (USIM) application
http://www.3gpp.org/ftp/Specs/archive/31_series/31.102/31102-c20.zip

The point of mentioning the abve standards brings the discussion back to an individual's understanding vis-a-vis the device being used against the target "DUT" and not leaving it to a machine to do the thinking.

Furthermore, it is essential to remember who is actually being 'caught in the net' under proposed changes. A response to a consultation I submitted some years ago raised the observation that if legal aid was to change regarding witness/expert fees where do organisations and scrutiny of the rules come into it where evidence is submitted directly e.g. from a mobile network operator. Never got a response on that one, but it really ought to be a fundamental requirement to identify who the FSR thinks should be excluded from his/her executive powers being imposed?

REMEMBER - DATA IS DISPASSIONATE, SO SHOULD THE EXPERT BE

Evidence and Admissibility Part 2

Evidence and Admissibility Part 2

.

In the first discussion on Evidence and Admissibility that appeared in Part 1 there was reference made to the International standards ISO17025 and ISO17020 and their proposed use as laboratory standards suited to forensics and evidence - http://trewmte.blogspot.com/2010/06/evidence-and-admissibility-part-1.html . In Part 2 further reference is made to ISO17025, the Forensic Science Regulator and UKAS and the Diplomas and their potential future influence dealing with evidence and admissibility. The purpose of making reference to these matters allows comparison to made between past technical, forensics and evidential events and potentially where the future lies when dealing with mobile telephone evidence.
.
The FSR (Forensic Science Regulator) as we know has identified a standard for laboratories that is said to be directly applicable to the field of digital mobile telephony, which is ISO/IEC 17025. To recap on background information; this is the main standard used by testing and calibration laboratories. This standard was formally known as ISO/IEC Guide 25. This standard contains many commonalities with QA standard ISO9000. When the second release of ISO17025 occurred in 2005 it did so based upon the agreement that its quality system wording was closely aligned with IS09000 version 2000. Readers may remember I referred to adopted principles in ISO17025 and QA standards in my previous posting Evidence and Admissibility Part 1.
.
So why is ISO/IEC 17025 applicable to all experts and examiners? Significantly, unlike ISO9000 and BS5750, this standard introduces the concept of, and the requirement for, 'competence' of the individual to be included into the equation because the standard applies directly to those organisations that produce testing and calibration results. The fit for purpose requirement of tools is set out under a different requirement, which is 'validation'. In order for 'tools' to remain valid equally relies upon the individual's competence, and at minimum should be:
.
a) Possess the highest competencies or attain them if the organisation is to rely on the individual to work with a tool;
.
b) To have been independently assessed by long-term experienced and skilled individuals as opposed to tool suppliers merely approving to get the sale of a tool;
.
c) Know what the 'tool' does;
.
d) Have the skills to assess whether the 'tool' is technically failing or systemically flawed in its operation;
.
e) To be able to maintain the 'tool's' performance - equally means maintaining an individual's skills;
.
f) To know and understand the requirements of future-proofing and sustainability.
.
How can the above be achieved? For sometime there has been a call for Mobile Phone Forensics Degrees but the Universities failed to take up the proposition. From enquiries made into this matter, it became clear the Universities, despite producing MSCs with one or half of one module containing something on mobile phone examination, the Universities do not have the competencies and skillsets for the work, nor could they identify the materials needed for the degrees or have the ability to assess student competence. That arises because the area is a specialism, thus requires a specialist approach and therefore this is no poor reflection on the academic brillance of Universities. So in this regard it makes no criticism of the Universities for not offering every specialism in the marketplace. This is why the Diplomas (http://trewmte.blogspot.com/2010/05/diplomas-mobile-telephone-evidence.html) have been introduced to solve this particular dilemma in the marketplace given the tens of hundreds involved with this field of distinction. The Diplomas provide a stepping stone to recognising 'competence', which currently examiners are being denied. Moreover, the modules have been prepared in the Diplomas to follow a similar structure to the way in which a student would undertake an MSC.
.
Another area where the Diplomas assist is in relation to cost. Currently MSC degrees cost between £3K to £7k per annum. The latest reported news in the media is that these tuition fees are set to increase even higher, maybe even skyrocket. The Diplomas work the other way around, the more modules you undertake the lower the costs become due to the way the discounts work. The encouragement to undertake the Diplomas therefore doesn't just include imparting real-world knowledge and experience passed on by seasoned professionals, but incentive discounts offered by uptake of the modules enables the securing in-advance of the professionals time which otherwise wouldn't be possible. Furthermore, and at the same time, they offer affordability to students.
.
Why is the way forward given above a better way than the current status quo? Neither the Forensic Science Regulator (FSR) or UKAS can approve the 'competence' of individuals for each particular field of forensic /evidential distinction. Universities as we know do not have the skillsets necessary to rollout degree courses in mobile telephone evidence. Specific to our field of distinction, the Mobile Telephone Examination Board (MTEB) was set up specifically to address peer review; the Diplomas are one way to gain peer review leading to recognition through the MTEB. The foundation to each of the modules in each of the Diplomas map principles to be found in simulated annealing for deterministic and random events that are often understood to be the basis of how evidence from mobile telephone usage occurs. The Universities gain from Diplomas as they work in harmony with them to aid students find the academic path to follow for a specific subject rather than the current way of devaluing and diluting other sciences and forensic fields to build up modules introduced into hybrid technology degree courses. So we need to understand the influences and impact of the quality or state of existing in or assuming different educational forms (generically speaking, polymorphic).
.
The future of mobile communications and its impact on society will grow even more than hitherto it has. The driving forces for this can be traced as much to remarks on future Global economies by industry illuminaries such as the head of LM Ericsson, the largest telecommunications manufacturer, when he said the future is "wireless" as much to the growth in smartphones and wireless smart devices and the newer wireless transmission and protocols, functionality and applications introducing diversity in the wireless world that looks set to surpass the cable world. However, to meet those exciting and progressive markets, Business can no longer afford to wait for degree students to flock out from Uni after 3 years and then spend another 1-2 year/s bringing them up to speed in the fast-changing wireless environment.
.
An example of fast-changing wireless environment which identifies sustained regular change can be found in the wireless standards used nationally and globally:
.
_____________________________________________________________
3GPP specifications
The term "3GPP specification" covers all GSM (including GPRS and EDGE) and W-CDMA specifications. The following terms are also used to describe networks using the 3G specifications: UTRAN, UMTS (in Europe) and FOMA (in Japan).
.
Revised versions of many of these specifications are produced up to four times a year following the quarterly TSG plenary meetings (TSG GERAN meets five times a year.)
_____________________________________________________________
.
The Diplomas take all of those conditions above, turn them around and re-work them. Students pursue the business goals first, being effective and self-managed at work, and then use the academic route to refine and hone particular, finite skills to optimise the end result for the benefit of the business. This model can assist Universities to get students wanting to achieve the technical hardcore goals in order to develop products, systems and services tailored from the diversity of the business enterprise and working environment. This can be so because the student's constructive thought process is based upon real-world exposure, not school-to-university idealogical observations. Moroever, there are examiners already working in the field who (a) have not had a fair opportunity to have their competencies recognised by (b) their peers and in the community. The Diplomas remove that unfair stigma for those mobile phone examiners being treated as second class.
.
So what are some of the future evidence and admissibility issues to be considered?
.
1. Recognised competence so reliance is NOT solely placed upon the tool; thus the removal of "push-button forensics" used to side-step the need for competency
.
2. Validation of tools will come about (thus side-stepping compliance is unavoidable). Indeed one particular tool developer agrees validation is entirely achieveable. So this is a step forward for validation, but again does not of itself qualify the 'competency' of the individual if trained to use the tool beyond gaining exposure to the tool's operation and use, but not the detail of the subject matter of the science, the technology using the science or, indeed, interpretation of outcomes or data.
.
3. Item 2 (above) is really a further qualification as to why the FSR and UKAS do not appear as candidates to qualify/recognise individual 'competency' in the subject matter or science.
.
In the next part, Part 3, the discussion deals with specific technical issues and their impact on evidence and admissibility.

Forensic Science Regulator Report

Forensic Science Regulator Report

.

Did you read this document from the Forensic Science Regulator dated July 2009?

.

Online Google Document:

http://docs.google.com/gview?a=v&q=cache:byViwdsmrFQJ:police.homeoffice.gov.uk/publications/operational-policing/practitioner-reg-summary%3Fview%3DBinary+Practitioner+Registration+Consultation+Responses+Summary&hl=en&gl=uk&sig=AFQjCNF-6wcm-lxTM8_6lhpF-VAIPRC1rw

.

Download .pdf

http://police.homeoffice.gov.uk/publications/operational-policing/practitioner-reg-summary?view=Binary

.

What are your thoughts?

.

Discussion Link

http://www.forensicfocus.com/index.php?name=Forums&file=viewtopic&t=4801

What is happening with Digital Forensics UK

What is happening with Digital Forensics UK
.

For those of you that did not know, which is probably most of the digital forensics community in the UK, as I didn't know, until today 26th February 2009, the Forensic Science Regulator is having a conference in Birmingham on the 31st March 2009 to be held at the International Convention Centre (ICC), Broad Street, Birmingham, B1 2EA. The aim of the conference is for the purpose "As part of the consultation process, the Regulator is to hold a conference for digital forensics practitioners as a group of experts to focus on the standards and to provide opportunities for detailed feedback."

.

Disappointingly, the closing date to submit applications to attend was the 23rd February 2009. Regrettably, there is no notification at the Forensic Science Regulator's website and no notification to the established expert bodies in the UK about the conference. Instead, the matter was left to a small, but interesting, digital forensic forum that does not represent, and nor is it the voice of, the digital forensics community as a whole in the UK, to circulate the conference notice but sent to those forum's members only.

Forensic Science Regulator

Forensic Science Regulator

.

I would urge all of you who examine mobile telephones and computers that if you want to do public sector work for evidence in civil and criminal proceedings in the UK and you are not on the Forensic Science Regulator's (FSR) list of approved suppliers then you may not get any work at all.

.

Dr Chris Pamplin from the UK Register of Expert Witnesses has presented some open discussion on the FSR's report.

.

One recommendation is the demise of the Council for the Registration of Forensic Practitioners (CRFP)

.

Here is the weblink and go to the section on Understanding the Issues and read all the threads...very informative.

.

www.jspubs.com/surveys/fsr0901/

.