Testing Security Products

ASIO and security product testing

One of the functions of ASIO by its legislative charter (the ASIO Act 1979), is providing protective security advice to Government. This includes advice on physical, administrative and personnel security. The advice is normally channelled through agency security advisers (ASAs) who are the officers within each Commonwealth department and agency responsible for advising agency management on security arrangements.

Physical security advice

The physical security advice that ASIO provides often includes recommendations for protective security products such as alarm systems, locks and barriers, security containers and many other items. It is vital that such products are known to have met high specifications of reliability and ease of use - for in the field of physical and national security, the consequences of technical failure or human error may be irreversible or intolerable. Where human life is at risk, or national interests at stake, security advice needs to be able to rely on products whose worth has been proved through a comprehensive testing and evaluation program.

ASIO's approach

When a client department or agency approaches ASIO for protective security advice, they can expect to go through an extensive consultation process. Because every site and set of circumstances is different, ASIO must in each case determine what sources of harm may threaten the client's security, as well as the consequences of a security breach and the level of risk. Then ASIO must assess the client's vulnerability. When it comes to assessing the vulnerability of the various physical security measures which might be applied, ASIO makes judgments by means of a comprehensive test and evaluation program.

ASIO's test programs enable reliable and cost-effective security equipment to be identified for use by government departments and agencies. All such equipment must meet strict requirements to complement the Commonwealth's Protective Security Manual (PSM), which sets out standards and procedures for the protection of national security and protected matter. Products which meet the test criteria are included in the Commonwealth's Security Equipment Catalogue(SEC).

Security Equipment Catalogue

ASIO publishes a revised edition of the Security Equipment Catalogue approximately every two years, on behalf of the Government's Security Construction and Equipment Committee (SCEC), which is the oversight body for administrative and physical protective security matters. The aim of the catalogue is to help agency security advisers and government-accredited technical staff and consultants to select appropriate products in support of security installations.

The Security Equipment Catalogue is a RESTRICTED circulation document, available to Commonwealth and State government agencies through subscription only (see contact details). The low-cost subscription also buys updates, working notes, protective security circulars and a newsletter, to keep subscribers up to date with new or superseded products and developments.

ASIO's testing arrangements

ASIO carries out its test program at the ASIO's Security Equipment Test Site (ASETS) outside Canberra, ACT. The site was established in 1989 and has the following facilities:

a building specifically designed for the testing of internal volumetric and perimeter detectors, alarm panels, access control systems, locks and other physical security devices;
an open-air test area for testing of perimeter intrusion detection systems, communications links, fences, closed circuit television, video motion detectors, lighting and deployable security systems;
monitoring facilities for the open-air site; and
a forcible attack facility for testing of barriers, locks, doors, and glazing.

In some circumstances, however, ASIO may conduct testing at client premises.

The work of the ASIO security equipment test site (ASETS)

ASIO's personnel at the ASETS are engineers and technical officers. Their job is to select equipment for testing, prepare the annual test and training program, set performance standards, test specifications, conduct or arrange testing and evaluation, and coordinate the test results. They review national and international standards for security equipment and systems and they serve on appropriate standardisation committees. They also liaise with overseas testing authorities so that appropriate test results can be shared, avoiding wasteful duplication of effort.

ASIO maintains a watching brief on technological developments, encourages product development in Australian industry, and supports research to meet known deficiencies in security.

As well as running the test program, ASIO uses the site to train government security staff in the use of various security products. Arrangements can also be made to permit companies to train or brief clients, or their own installation staff.

Status of ASETS

Testing Area

The large internal testing area at ASETS may be configured for testing a variety of products.

The ASIO test site is not accredited with the National Association of Testing Authorities (NATA). This is because ASETS specialises in on-site tests which are performance orientated against varying threats and threat intensities, rather than quantifiable standard-linked testing. In many cases, ASETS has to develop specific test specifications for individual pieces of equipment or for site-specific applications. Where NATA-accredited testing is required against either ASIO or public standards, ASIO liaises directly with government NATA laboratories, such as the Scientific Services Laboratory in Melbourne.

Access to the test site

Visits to ASETS are by arrangement only (see contact details at the end of this booklet). People eligible to visit are generally Commonwealth and State government personnel and staff of government business enterprises. Representatives of companies with equipment under test or installed on the test site may also arrange visits, accompanied by potential clients if appropriate. All such commercial visitors are escorted, to maintain the confidentiality of competitors whose equipment may also be undergoing testing at the time.

The testing process

How ASIO's testing differs from standard testing

There are two principal aspects to the standard testing of equipment:

quantifiable laboratory testing where the equipment can be tested for reliability and performance under specific environmental or site specific areas;
qualitative testing where the equipment is tested to determine predicted performance against the variety of attacks which may be expected against various facilities. The equipment may be stand-alone or installed in an integrated configuration.

Laboratory testing or manufacturer quality assurance accreditation is invariably undertaken on quality products, and this is a necessary exercise. These tests, however, generally apply to equipment under controlled and predictable conditions, and activated by predictable stimuli.

In the real world, security products must be able to stand up against variable conditions and unpredictable attacks by knowledgeable adversaries whose performance can be affected by physiological, psychological and environmental influences at the time. ASETS provides facilities for assessing and being able to predict performance under varying conditions and different scenarios. ASIO testing thus adds an extra (and necessary) dimension to the quality assurance provided by the manufacturer.

Test Site

The test site is extensive, allowing products such as security fences to be tested under varying environmental conditions.

Tests and criteria for government high security

Because of the considerable resources required for the test program, only those products which are required to meet identified needs of government are considered for testing. The tests and criteria to be met before security equipment is considered suitable for government high security applications are as follows:

Bench test

A bench test is applied to assess fit for purpose, quality of manufacture, documentation, et cetera.

Laboratory testing

Laboratory testing is normally mandatory before equipment will be considered at ASETS for testing for high security applications. Laboratory testing ensures the equipment complies with the manufacturer's statement of performance and that safety and other regulatory considerations (such as electromagnetic compatibility) are inherent. Overseas standards and test results may be accepted. Otherwise, local testing to ASIO standards by an Australian NATA-certified laboratory may be required. The laboratory standards may be selected from overseas accreditation agencies or developed by ASIO.

Walk-testing

Walk-testing of electronic intrusion detectors is usually carried out at ASETS to ensure that the equipment will perform as required in a real-time environment against the appropriate target. Walk-testing also indicates the sensitivity settings that would be required in an operational installation. For some equipment (such as doors and locks), this may include reliability testing under extended usage.

False/nuisance alarm rate testing

When the performance settings have been determined, the equipment is left to operate to assess the false and nuisance alarm rates. The equipment is monitored along with the field of view of the equipment and the environment. This allows evaluation of the susceptibility of the equipment to unwanted stimuli. Alarm monitoring, monitoring of weather conditions, and closed circuit TV coverage 24 hours a day assist in determining the causes of any alarms.

Testing

An ASETS staff member acts the part of an intruder for a manufacturer's video promoting a new security alarm panel.

Defeat testing

Defeat testing determines ways and means of defeating the equipment, as well as the resources required and the delays that could be expected in the intrusion. All systems are vulnerable, and defeat testing is essential if a security system involving barriers and complementary devices is to be effective. Understanding the skills, knowledge and resources which an intruder would need to break the system enables design specifications to be improved upon for later development.

Evaluation of supportability

On imported products, an evaluation of the Australian supplier may be required to confirm the level of support available for the product, either locally or from the overseas manufacturer.

How products are selected for testing

Equipment to be tested may be selected by ASIO to serve clients under its protective security advice program, or it may be selected on the basis of a determination by the Commonwealth's Security Construction and Equipment Committee (SCEC). This is an interdepartmental committee whose members represent major agencies concerned with public protection, national security, or the protection of valuable national assets. If SCEC determines that Government has a functional need for a product, it becomes the responsibility of a SCEC working party to decide on likely products for testing. SCEC has three working committees:

Administrative Security Equipment Working Committee - responsible for matters relating to copiers, destructors, carriage of documentation and assets, security seals, personal identification passes et cetera;

Electronic Security Equipment and Systems Working Committee - responsible for matters relating to alarm panels, access control, closed circuit television, internal and external intrusion detectors, transmission systems and other electronic based security systems;

Security Hardware and Construction Working Committee - responsible for matters relating to locks, barriers, security containers, secure rooms or vaults and building construction.

Membership of SCEC working committees is restricted to those agencies having the technical expertise to contribute to the equipment evaluation process, or to agencies which have a significant investment in the areas of interest. Feedback from user agencies is essential in validating the ASETS test program and enables continuing assessment of equipment reliability and company support.

Compliance

Products may be tested to confirm their compliance with ASIO design specifications.

Submitting equipment for test

The decision to test specific products (whether recommended by SCEC or chosen by ASIO with client needs in mind) is made by ASIO after consideration of the test resources available. ASIO may solicit equipment for test to meet known or anticipated needs. ASIO may also consider testing equipment if a deficiency in an endorsed security product is known to exist. Equipment suppliers or manufacturers can offer equipment for consideration. Priority is given to equipment needed to satisfy national security concerns.

Information for manufacturers and suppliers

ASIO's test site offers companies a permanent site for installing and demonstrating their equipment - a significant advantage over having to demonstrate their product in a number of temporary installations nationwide. In addition, a permanent installation is likely to enable higher levels of performance and reliability than equipment set up in a temporary site. Companies may bring potential clients to the site, and ASIO staff will assist in conducting demonstrations and in making appropriate test data available.

Care of equipment

Equipment is expected to be supplied free of charge by the companies for testing. Although all due care and reasonable security is given to equipment, the Commonwealth can assume no liability for any loss, damage, or destruction to the equipment while under test. If loss or damage can be foreseen (for instance, in destructive testing), ASIO informs the company before testing begins. In such cases, company participation in the testing may be invited.

Long term monitoring

Generally, product that has been tested and endorsed is required to be left in place on site for long-term reliability testing. This enables ASIO to monitor performance and identify impending equipment problems before they materialise in operational sites. It also assists ASIO with evaluating equipment modifications and the suitability of the maintenance schedules in the Australian environment.

Site-specific testing

As noted, ASIO may conduct testing to determine the performance of equipment in a particular location. This may be to identify the most suitable equipment, or configuration of equipment; to test for improving performance; or to identify problems with already installed systems. In these cases, equipment may be given a site-specific endorsement.

Installation responsibilities

For small items such as internal intrusion detectors (where any number of companies could be expected to install the equipment), ASIO technical staff will carry out the installation, using the commercially supplied documentation. The quality of documentation, ease of installation, et cetera. will be assessed as part of the equipment evaluation.

For larger equipment or systems, where the quality of installation and site preparation is likely to effect the short-term and long-term performance of the equipment, all installation tasks are the responsibility of the company. The quality of the installation is also considered to be an assessable aspect of the evaluation of the equipment.

In some cases, endorsement may require that installation be carried out by nominated companies only.

Responsibility for incorporating subsequent modifications or for carrying out scheduled maintenance is negotiable, depending on whether company or client responsibility is envisaged on operational sites.

Responsibility for bearing costs

All costs associated with the testing and accreditation of equipment are recovered from the company or requesting agency.

Australian companies seeking help with product development are likely to receive preferential consideration, particularly if the equipment has export potential.

Release of test results

A test report (which is a summary without detailed test data) will be provided to the company for dissemination at its discretion. Detailed test results will be released only at the discretion of ASIO. This restriction applies because detailed tests are open to misinterpretation unless all information on the test methods, test equipment and test results are available. However, data may be made available for discussions with the company or for partial release as part of company efforts to correct any found deficiencies.

Full test results may be presented to the members of the appropriate SCEC working committee as a basis for their recommendations to SCEC. Generally, however, the committee will present SCEC with only the test report or summary (without the detailed test data), along with recommendations for endorsement (with or without qualification). Products tested but not recommended will be submitted by listing to SCEC for recording purposes only.

Test reports may be listed in the Security Equipment Catalogue and be made available to agencies which have a genuine need for access to the data.

Working notes

Working notes may be compiled by ASIO staff during the process of testing, to supplement company documentation or to detail specific requirements for installing the equipment to satisfy SCEC requirements. They are intended primarily for use by technical personnel involved with system design, installation and maintenance. Working notes are listed in the Security Equipment Catalogue but are issued only on application, to ensure a distribution list can be maintained for amendment purposes.

Equipment certification

Certification of Testing documentation may be provided in some circumstances. However companies are not permitted to use endorsement in the Security Equipment Catalogue for public promotional purposes without the approval of SCEC. This is because many items included in the Security Equipment Catalogue are endorsed with some qualification as to their performance and applications, and if the promotional literature did not reflect this, it would be misleading.

Reference can be made to the Security Equipment Catalogue in promotions of products which have been endorsed without qualification. (This occurs only after extensive testing under different threats and environments and to a degree that indicates Government has a high level of confidence in both product and company reliability.) However, in each case the details must be negotiated with SCEC.

ASIO may impose a charge to provide for regular revalidation of such listed equipment.

Testing schedules

ASIO's annual test program is developed with consideration of the needs of ASIO clients, the concerns of SCEC, and the availability of ASIO resources. Every effort is made to conform to program test schedules. However, testing resources may sometimes need to be diverted to more urgent matters of national security, and ASIO reserves the right to suspend or postpone testing in such circumstances.

Assistance to industry

Australian designers or researchers working on projects which may result in product with security applications are invited to contact ASIO during product development. ASIO staff at ASETS will be happy to provide advice on the requirements for government high security equipment.

For more information

If you wish to find out more about ASIO's annual test program, or the ASIO security equipment test site, or how to subscribe to the Security Equipment Catalogue, please contact us at the following:

Telephone (02) 6234 1217

Fax (02) 6234 1218

International +61 2 6234 1217

Mailing details are:
T4 Protective Security
GPO Box 2176
CANBERRA 2601

Courier delivery address:
T4 Protective Security
ASIO Central Office,
cnr Russell and Kelliher Drives,
RUSSELL ACT

Testing Security Products

The work of ASIO's security equipment test site