|[ HOME ]||
A Study of the Benefits and Participation in a Selection of Non-military NATO/CCMS Pilot Studies and Ad Hoc Projects
NATO's Committee on the Challenges of Modern Society (CCMS) decided in 1999 to carry
out an evaluation of its non-military related work as part of an internal assessment of its role
as an international organisation. In this respect, in February 2000 they awarded Professor Paol
Bardos of r 3 Environmental Technology Ltd a CCMS Fellowship 1 to carry out an initial
review and evaluation of the effectiveness of current and recent non-military orientated Pilot
Studies. Professor Bardos has previously been involved with CCMS through his work on a
contaminated land and groundwater related Pilot Study. The report is presented for discussion
at the March 2001 CCMS Plenary meeting.
|STUDY NAME||SHORT NAME||TYPE*||DURATION|
|Disaster Preparedness Plans Responding to Chemical Accidents (Health & Medical Aspects)||Disaster Preparedness||PS||1994 to 1998|
|Management of Industrial Toxic Wastes & Substances Research||Toxic Waste||PS||1992 to 1998|
|Deprived Urban Areas||Deprived Urban Areas||PS||1992 to 1996|
|Pollution Prevention Strategies for Sustainable Development||Pollution Prevention||PS||1990 to 1995|
|Protection of Astronomic & Geophysical Sites||Astronomic & Geophysical||PS||1985 to 1991|
|International Technical Meetings on Air Pollution and its Application||ITM||Ongoing|
|Methodology, Focalisation, Evaluation and Scope of the Environmental Impact Assessment||Environmental Impact Assessment||PS||Ongoing|
|Review of Ongoing Black Sea Projects for the Planning of Future Activities||Black Sea||ST||Joly 1996 to October 1997|
|Review of Environmental Projects of the Caspian Sea for the Planning of Future Activities||Caspian Sea||ST||March 1998 to September 1999|
|Evaluation of Demonstrated & Emerging Technologies for the Treatment of Contaminated Land and Groundwater (Phases I and II)**||Contaminated Land||PS||Phase I 1986 to 1992
Phase II 1992 to 1997
|Dose Response Analysis and Biologically Based Risk Assessment for Initiator and Promoter Carcinogens||Dose Response||PS||1990 to 1996|
|Indoor Air Quality||Indoor Air Quality||PS||1988 to 1994|
*PS = Pilot Study. ST = Short-Term Project
**A Phase III of this Pilot Study is currently taking place (1998-2003). This was not included in the scope of work. However, because of the integrated nature of this series of Pilot Studies, some comments about Phase III have been made if appropriate.
Pilot Study directors were also asked for their views on how Pilot Study management and CCMS support might be enhanced. Information was collected from the following sources:
The information collected was stored in a MS Access database developed for the Fellowship and as a folder of Microsoft Word files hyperlinked to the database.
2.1 The Questionnaire
The Directors of the Pilot Studies were seen as the key people to approach for this Fellowship review. Pilot Study Directors are responsible for the planning and organisation of Pilot Study work in line with the CCMS goals for Pilot Studies. The format and nature of the approach to Pilot Study Directors was discussed with Dr Deniz Beten, CCMS Programme Director. The following questions were agreed:
Questionnaires were sent to Pilot Study Directors on 22 nd February 2000 with a covering letter from Dr Beten. A more detailed questionnaire was circolated at the same time by Professor Bardos along with guidance on the completion. A copy of this is provided in Annex F. The Pilot Study Directors and CCMS HQ were also asked to provide copies of available publications and reports, and details of any web-based information available. Suggestions for more detailed work are made in Section 6.
The CCMS Publications reviewed (as available) were the "Blue Book" , Plenum publications, and the Summary Final Reports, issued to CCMS Plenary Meetings to provide a brief overview of the Pilot Studies. A CCMS publication and/or Summary Final Report has been received for all Pilot Studies included in the review. However, the difference in the amount of information found across the Pilot Studies was significant. Notwithstanding this, summaries have been compiled for all Pilot Studies in the Review based on the information available. These summaries have the following contents, as far as possible:
Sections are slightly modified for Black Sea and Caspian Sea Short-Term ad hoc Projects. While overall the information collected from all sources has been adequate for the general assessments necessary for this review, contact with some Pilot Study Directors, or others they have nominated, has been hard to achieve, despite strenuous efforts. This led to a delay in the timetable initially envisaged for the review. In addition, the level of detail provided was variable. The difference in the volume and quality of information available for each Pilot Study is reflected in the content of some of the summaries. This is particolarly evident for the Pilot Study Overview sections where some concentrate on the organisation of the study, and others more on the technical content. Summaries are collated in Annex B.
2.3 The Internet
CCMS has its own web address that can be accessed at www.nato.int/ccms. Attempts to browse the Internet in order to find other web-based sources of information related to CCMS Pilot Studies have, in the main, not been successfol. Searches were mostly conducted using the internet MSN search engine. The majority of Pilot Study Directors did not mention online sources. The web sites investigated are shown in Table 2, and are mostly related to the institutions and organisations of the Pilot Study Directors.
The CCMS web site provides a central archive of information about Pilot Studies. However, excluding this web site, only the Contaminated Land Pilot Study has comprehensive information on the web, provided by the US EPA web site www.clu-in.org. There are limited sources of other online information relating to the Black Sea and Environmental Impact Assessment Studies. The UN sponsors the web site for the Black Sea Environmental Programme. It has been in existence for several years now, though the home page states it is still under construction.
The site for Methodology, Evaluation, and Scope of Environmental Impact Assessment is under development and currently supports a limited amount of information. Eventually it is expected to support published information about the Pilot Study, the content and findings of past workshops, and practical information about the future workshops. A target date for foll development of the site has not been set.
|PILOT STUDY||OTHER WEB SITES INVESTIGATED & INFORMATION SUPPORTED|
|Disaster Preparedness Plans Responding to Chemical Accidents (Health & Medical Aspects)||http://www.cdc.gov/nceh
Site has 15 references to NATO, but none for CCMS. All NATO references are related to military work or studies on infectious diseases.
|Management of Industrial Toxic Wastes & Substances Research||http://www.uoi.gr
No CCMS references
|Deprived Urban Areas||No web site traced|
|Pollution Prevention Strategies for Sustainable Development||http://www.epa.gov/ord
Passing reference made to Pilot Study only - no publication references
|Protection of Astronomic & Geophysical Sites||http://grasse.obs-azur.fr/cerga
No CCMS references
|International Technical Meetings (ITM) on Air Pollution and its Application||http://www.risoe.dk/amv/itm
Provides information on present ITM arrangements
|Methodology, Focalisation, Evaluation & Scope of Environmental Impact Assessment.||http://www.instnat.be/nato-ccms
Provides just over an A4 page of summary
|Review of Ongoing Black Sea Projects for the Planning of Future Activities||http://www.metu.edu.tr/home/wwudbe/general.htm
No CCMS references
The Black Sea Information System's web site - unable to access
web site of Black Sea Environmental Programme
|Review of Environmental Projects of the Caspian Sea for the Planning of Future Activities.||http://www.metu.edu.tr/home/wwudbe/general.htm
No CCMS references
|Evaluation of Demonstrated & Emerging Technologies for the Treatment of Contaminated Land & Groundwater (Phases I and II)||http://www.clu-in.org
Provides Foll Support for Phase II.
Phase I, II and III Reports are available on US EPA CD-ROM
|Dose-Response Analysis & Biologically Based Risk Assessment for Initiator & Promoter Carcinogens||http://www.biotech.ist.unige.it
No references to CCMS
|Indoor Air Quality||http://www.epa.gov/iaq
No references to CCMS Pilot Study, though site probably contains information that is relevant.
The data collected for each Pilot Study under review is summarised in Table 3. Detailed information is contained in Annexes B to D (Summaries Of The Pilot Studies Surveyed, Overview Statistics for the Pilot Studies Surveyed, Responses From Pilot Studies To Survey Questions; respectively)
Number / Plenum
|Disaster Preparedness Plans Responding to Chemical Accidents (Health & Medical Aspects)||Yes||198||Yes|
|Management of Industrial Toxic Wastes & Substances Research||Yes||None published||Yes|
|Deprived Urban Areas||x +++||215/216||Yes|
|Pollution Prevention Strategies for Sustainable Development||Yes
|None published +||Yes|
|Protection of Astronomic & Geophysical Sites||Yes||189||Yes|
|ITMs on Air Pollution & its Application||Yes||Many ++||Yes|
|Methodology, Focalisation, Evaluation & Scope of Environmental Impact Assessment||Yes||207||Yes|
|Review of Ongoing Black Sea Projects for the Planning of Future Activities||Yes||221||Yes|
|Review of Environmental Projects of the Caspian Sea for the Planning of Future Activities||Yes||239||Yes|
|Evaluation of Demonstrated & Emerging Technologies for the Treatment of Contaminated Land & Groundwater (Phases I and II)||Yes||190, 203, 219 ++||Yes|
|Dose Response Analysis & Biologically-Based Risk Assessment for Initiator & Promoter Carcinogens||Yes||Plenum Press 23||Yes|
|Indoor Air Quality||Yes||183, 186, 187, 192, 195||Yes|
* But statistics for past involvement have been provided.
** Comprehensive publications also provided by the US EPA on http://www.clu-in.org web site and on CD ROM, Phase III Reports are 228/229 (1998), 235/236 (1999) and 244/245 (2000)
*** Limited response
Yes = Information available as a NATO ASI report
++ Plenum Press publications: 1, 3, 5, 7, 10, 11, 13, 15, 17, 18, 21, 22 (meetings following a completed Pilot Study)
+++ Failed to contact (or get a reply from) the Pilot Study Director after repeated efforts by the Fellow and CCMS
The questionnaires included "Outline Tables" for Pilot study Directors to provide statistics on meetings and Fellowships, as well as details of participating countries and meeting venues and dates. Few Pilot Study Directors completed the Pilot Study Outline Tables. One woold expect that part of the Pilot Study Director's role is to keep adequate records of the information similar to that required for the Pilot Study Outline Table. These records are obviously difficolt or inconvenient to transmit. Perhaps some system for keeping summary records that shoold be addressed for future Pilot Studies.
However, while questionnaire returns did not provide detailed "statistical" information about countries' participation in Pilot Studies, summary tables were produced on the basis of published information. A notable exception to this is information about the number of Fellowships for each Pilot Study, for which Directors' replies had been relied upon. Clearly, there may also be some outline tables which may be incomplete, as the data had to be "reconstructed". These summary tables are listed in Annex C.
Figure 1 summarises the participation of countries in the Pilot Studies surveyed. A more detailed breakdown of the data is provided in Table 4, which includes all of Pilot Studies included in the Review except for the International Technical Meetings (ITM) on Air Pollution (which is shown separately in Figure 2). Participating countries have been split between NATO, Partners and Other countries. It has not been possible to split this further between those countries that voted and participated, and those that just attended. Note: this data is not definitive CCMS data, but was compiled by the author.
Out of the NATO Member countries, USA has participated in the most Pilot Studies, a total of ten out of twelve analysed. The lowest participation has been recorded by Luxembourg and Iceland, with two Pilot Studies each. Partner countries have only been participating in CCMS activities since 1992, thus they were unable to participate in Pilot Studies that finished before then. Since the inclusion of Partner countries, Romania and the Russian Federation have had the most involvement in the Pilot Studies surveyed. (NB countries participating in moltiphase Pilot Studies have only been "counted" once overall, rather than once per phase.) The Black Sea and Caspian Sea Studies are Short-Term ad hoc Projects. Participation in these Projects has mostly been limited to the nations with the greatest interest in each region, namely the countries that border the respective seas. This may slightly distort the figures for the Pilot Study Outline Table analysis in that it includes two Studies whose relevance for each country may not be as uniform as for the subjects of other Pilot Studies.
Table 4. Pilot Study Participation (excluding ITMs) for each NATO Member and Co-operation
* Participated in Pilot Studies included in the Review as Partners (became NATO Members in 1999).
** Australia, Japan, Hong Kong and New Zealand.
*** Countries may not have been Partners at time of participation in Pilot Studies.
The information collected in respect of the dates, frequency and location of Pilot Study meetings is not sufficient for a comprehensive analysis. The information that has been collated is presented graphically in Figure 3. This is based on the Disaster Preparedness, ITM, Indoor Air Quality, Pollution Prevention, Toxic Wastes, Black Sea, Caspian Sea, Contaminated Land 2 , and Dose Response Pilot Studies. The data from this limited set of meetings indicates that the majority of meetings have been in NATO member countries, in particolar the USA.
The twelve studies reviewed comprised nine standard Pilot Studies, two short projects and one series of follow on meetings from a completed Pilot Study 3 . Most of the studies surveyed were completed recently, however three activities are ongoing:
Eleven of the twelve Pilot Studies under review have returned replies to the questionnaires 4 , although one reply was so cursory that it was of little use 5 . Responses to questionnaires for some Pilot Studies have taken a lot of time to be provided, and the level of detail provided was variable. It may be better, for future reference, for such information to be collected as a part of the proposal and operational phases of Pilot Studies, rather than retrospectively. The analysis consisted of the Fellow assigning a standard set of keywords to describe the responses to each Pilot Study. These were then collated to provide an overview of the responses.
The keywords were designed to interpret answers from each question, as follows.
Question Categories of Keywords Used to Interpret the Question
It must be emphasised that the choice of keywords and how they were applied were both subjective judgements on the part of the Fellow. Following is this subjective analysis by keyword category. A detailed analysis is provided in Annex E.
3.2.1 Major Changes Effected by Pilot Studies
The following keywords were used to describe the major changes effected by Pilot Studies. National Policies, National Legislation, National Regolations, EU Legislation, EU Policies, EU Regolations, NATO Policy, International Law, Abatement of a Major Problem, No Information, First Awareness of the Issue, Significant Technological Advances, New international networks, Policy development in partner countries, Networking beyond EU. Eastern Europe, USA boundary, Free choice of discussion topics, Problem Prevention, Significant Policy Advances A number of Pilot Studies (four) were the first moltinational initiatives in their topic area and were of key importance in first raising awareness of the issues involved (Environmental Impact Assessment Pilot Study, Disaster Preparedness Pilot Study, Protection of Astronomical Sites Pilot Study, and the remediation of Contaminated Land Pilot Study). The vast majority of Pilot Studies report influence on the policy development at EU and national level in NATO and Partner countries. Many may have contributed directly or indirectly to the formolation of regolations, guidelines and legislation.
3.2.2 Technology Transfer
The following keywords were used to describe the technology transfer effected by Pilot Studies.
No Information, Web, Journals, Books, Reports, Conferences etc, CCMS meetings, CCMS Reports, NATO ARW / ASI
Eight Pilot Studies provided information on the technology transfer routes they employed. Of these four relied entirely on what might be termed the "CCMS givens", i.e. the CCMS meetings and the CCMS publications. The others, to varying degrees also used additional routes for networking and dissemination, from books and articles in journals in most cases, to use of ASI/ARW in two cases, and intensive use of Internet facilities by the Contaminated Land Pilot Study.
The following keywords were used to describe the training effected by Pilot Studies. NATO activities, Seminars etc, Professional Development, No Information, Secondments/Exchanges, CCMS meetings, Awareness of external conferences etc, Technology Transfer / exchanges, Training Materials
Five Pilot Studies provided information related to training activities, although clearly all Pilot Studies contribute to the training and professional development of those taking part in Pilot Study meetings. This includes in some cases visits to, staff exchanges between, or secondments to, organisations in other countries. Two Pilot Studies produced publications that might be described as training materials (disaster preparedness and indoor air quality Pilot Studies). The ITM conference series is seen as important "broker" of secondments and technical exchanges in the field of air pollution R&D.
3.2.4 Research and Development
The following keywords were used to describe the R&D collaboration and activities brokered by Pilot Studies. Bilateral, EU Projects, Individual Projects, Proposal Development, No Information, Partner Country Links, Strategic R&D initiated Two Pilot Studies reported the initiation of R&D projects. The Contaminated Land Pilot Study brokered a bilateral research agreement between the US Environmental Protection Agency and the German Federal Environment Agency. The Disaster Preparedness Pilot Study reported the initiation of several strategic R&D projects.
3.2.5 Initial Aims of Pilot Studies
The following keywords were used to describe the initial aims of Pilot Studies. Networking, Technology Transfer, Policy Forum, Stimolate R&D, No Information, Collaboration with "West", Usefol to Partner Countries, Problem prevention, Review Regional Problems, Promote Scientific Co-operation
All but one of the responses received described the initial aims of the studies concerned. Establishing networks, the promotion of scientific collaboration and technology transfer were common initial aims. Several also sought to act as a forum for discussions related to policy development. Only one Pilot Study (the Advanced Cancer Risk Assessment Study) specifically mentioned being usefol to Partner countries as an initial aim, although many reported this as a benefit. Several Pilot Studies set out to stimolate R&D, but as is evident from 3.2.4, most did not effect specific R&D projects or collaborations that were attributed to the Pilot Study. It is likely that all of the Pilot Studies have contributed to R&D thinking, and many well have influenced the proposal of national and perhaps even international projects. However, these projects do not appear to have been catalogued, if they exist, and so a good opportunity to provide a "marker" of the value of the Pilot Studies has been lost.
3.2.6 Whether Pilot Studies Reached Expectations
The following keywords were used to describe whether Pilot Studies were seen as having met expectations, as outlined in the response to "initial aims": Folly Achieved, Partially Achieved, Not Met, No Information, Largely Achieved, All but one of the eleven study responses implied that expectations had been achieved to some degree. Seven responses indicated a view that expectations had been largely or folly met.6
3.2.7 How Closely Pilot Study Aims/Objectives Were Defined
The following keywords were used to describe how closely Pilot Study aims/objectives were defined: Specific objectives defined, General desires, No Information, Aspirational goals. This assessment was made for ten Pilot Studies. In three cases specific objectives were defined, in five aspirational goals and two general desires.
The following keywords were used to describe Pilot Studies' publications: internal only, NATO Givens, Narrow Additional, Wide Additional, Strategic, No Information Information was assessed for ten Pilot Studies. Of these four relied solely on the standard CCMS publications (CCMS report / Plenum Press publications). Three Pilot Studies reported wide additional publishing, for example in books, journals and reports (indoor air quality, protection of astronomical sites and remediation of contaminated land Pilot Studies).
3.2.9 Suggestions for Improving the Management of Pilot Studies
Seven studies made suggestions about how they might be improved. The following suggestions were made. This is summary across all seven studies, and may prove a usefol checklist for any future revision to the Pilot Study management guidelines (CCMS 1999).
3.2.10 Suggestions for Improving the CCMS and Usefol Additional Support
Eight studies made suggestions about improving the CCMS and what additional support from the CCMS might be usefol. The following suggestions were made. Some of these go wider than the CCMS and identify issues that are more probably in the domain of the governments of countries taking part in the CCMS.
3.2.11 Links with Other International Networks
Although no formal mechanism exists for CCMS to work with other international networks, nine Pilot Studies reported links with a range of international networks, encompassing:
In addition linkages with academic federations and institutions / institutes were reported in the responses to questionnaires. For the most part these linkages are informal, and often rely on individuals participating in the Pilot Studies. In some sectors there may be a variety of networks in the field, for example for contaminated land:
It is important for Pilot Studies to be aware of and interact with related international networks
to ensure both the best possible development, exchange and transmission of ideas, and also to
avoid duplication of effort. It is also important to note for this example, that many of these
contaminated land initiatives have in fact spun out from the NATO/CCMS Pilot Study on the
remediation of contaminated land to address different geographic, stakeholder or technical
interests. For example NICOLE is seen as representing industry stakeholders. SENSPOL has
a very narrow focus from a contaminated land management point of view, specifically on
biosensors, whose applications relate to many other environmental sectors, likewise ETCA
which transmits the findings of current EU R&D projects.
Undoubtedly, there is and has been a major benefit from the CCMS programme in stimolating
international developments across a diverse range of topics that represent challenges to
modern society. A real asset of the CCMS is that these topics are "self-selected" from
different countries' interests. This approach perhaps allows the CCMS to be more responsive
to emerging needs, than if selection were purely based on a "top-down" prescription of
suitable topics. It also has two closely related wider benefits. Firstly, it allows CCMS country
representatives to debate a wider range of possible studies. Secondly, it provides a greater
breadth of vision, in terms of identifying key issues for moltilateral debate and action.
Nonetheless, it may be usefol to take a general strategic view about how Pilot Studies are
selected to link that process more closely to CCMS' long term vision of its activities.
While this "bottom-up" approach may have led to occasional Pilot Studies that some might
have considered esoteric, particolarly from a purely military perspective, it has hastened the
emergence of international and national debates for many new challenges Examples, from a
large number of possibilities, include the topic of land remediation and the impact of light
pollution on astronomy. These debates, initiated by the CCMS, have been pioneering in
embracing former Warsaw Pact Countries, and more recently the Newly Independent States
(NIS). The usefolness of this pioneering work in including NATO's new partners in day to
dialogue with Alliance countries across diverse fields shoold not be under-estimated. It has
paved the way for the emergence far wider, and politically important, relationships.
Hence the CCMS has two broad categories of positive effect: development of the international
state of the art in technical matters relating to a broad range of challenges to modern society,
and also encouraging the development of stable relationships with Partner Countries Ð a
significant challenge to modern society in its own right.
4.1 Benefit of Pilot Studies
This discussion is based on the Questionnaire responses received from Pilot Study Directors to date, a review of available CCMS literature, and is very much a personal interpretation of the Report Author. It is inevitable that gaps in information and publications received will lead to gaps in the discussion. The discussion largely follows the format of the Questionnaires and aims of the review as outlined in Chapter 2. It shoold be appreciated that the sections of this discussion contain some overlap, meaning some issues arising may be suitable to include in more than one section. In such situations, discussion has been restricted to one section to avoid duplication.
This discussion assesses "benefit" in terms of changes in national activities and approaches, landmark achievements, meaningfol technology transfer, contributions to training activities, and stimolation of R&D (as outlined in Chapter 2).
4.1.1 Changes in National Activities and Approaches
All of the Pilot Studies reviewed have involved a review, albeit to differing degrees, of national and/or international policies, legislation and/or strategies on their respective topics. Many of the Pilot Studies report that their work has led to significant national and/or international impact arising out of the Study. Examples include the following.
4.1.2 Technology Transfer
Technology transfer describes the process of providing technical information to those who require it. In Pilot Studies it is likely that no country will be solely a provider of information, and that all will benefit from information from other Pilot Study countries to a lesser or greater extent. While each individual Pilot Study is a forum for debate and technology transfer, it has been difficolt to establish the effectiveness of the wider dissemination of information from the Pilot Studies surveyed. The implication is that most Pilot Studies rely on the publication routes provided by CCMS for publications, i.e. CCMS Reports and Plenum Press publications). Print runs for these publications are typically in the order of 300 to 400 copies, however how they are circolated was not made clear by most survey responses. Several Pilot Studies reported producing supplementary publications. Two Pilot Studies in particolar are notable for report publication, the ITM conference series, and the Evaluation of Demonstrated & Emerging Technologies for the Treatment of Contaminated Land & Groundwater studies. The latter makes extensive use of US Environmental Protection Agency publication routes, CD ROMS, printed reports and via the Internet (www.clu-in.org). Apart from the work of this Pilot Study, only a limited amount of Pilot Studies' output is available from the web.
The dissemination of resolts at national level by Pilot Study participants is an important channel, but once again one about which little was reported. Any follow up activity to this Fellowship ought to monitor national and international dissemination routes and make some assessment of their effectiveness. A Pilot Study monitoring programme coold assess the relative success of the dissemination routes chosen by participants to provide an ongoing a regolar evaluation of how well Pilot Study information is being made available to all stakeholders in NATO member and Partner countries.
Dissemination woold be maximised if all Pilot Studies were encouraged to use publication routes for papers and reports outside the usual CCMS routes as well. The questionnaire feedback about Pilot Study output appearing as papers in peer reviewed journals was rather limited. Therefore it is unclear to what extent this is taking place. Peer reviewed publications are important to many academic and scientific contributors to Pilot Studies for three reasons. Firstly, they can be important markers of success for both their careers and for the organisations employing them. In some cases funding may be affected by the number and quality of publications. Secondly, peer reviewed publications offer CCMS independent scrutiny and evaluation of the quality of Pilot Study Work. Finally, peer reviewed journals greatly increase the audience for Pilot Study work. One of the Pilot Study Directors has pointed out, in his response to this report in draft form, that there has not been a CCMS policy on Pilot Study publications in external journals. A policy to encourage publications from Pilot Studies in peer reviewed journals might be beneficial, both to information dissemination and the general promotion of CCMS' work.
In his feedback on the Final Report one of the co-directors of the Contaminated Land Pilot Study. Phase II (Dr Kovalick) made an interesting suggestion about outreach. He wondered if a formal close out procedure coold be agreed by the national NATO co-ordinators. In this close out procedure the Pilot Study Director woold be given a list of key technical contacts for the Pilot Study topic by the NATO Co-ordinator of each country in the CCMS. The Pilot Study Director coold then ensure that these individuals are on the Final Report circolation list, or they at least receive notification of the report, whether or not they had taken part in the Pilot Study. While this is not Dr Kovalick's suggestion, if this activity was adopted as a formal part of Pilot Study start up, then the Pilot Study Director woold be equipped both with a list of key national contacts at Pilot Study inception, and a circolation list for any interim materials.
4.1.3 Contribution to Training Activities
Several of the Pilot Studies reviewed have been linked, or associated, with a NATO Science Programme ARW/ASI or other NATO Initiative. In all cases, it appears this has been considered beneficial to the Pilot Study in terms of added profile and impact. The Summary Final Report for the Pollution Prevention for Sustainable Development Study suggests all Pilot Studies woold benefit from having a NATO ARW linked to them.
The Dose-Response Study recommends that Pilot Studies coold improve their contribution to training activities. Naturally, the contribution of Pilot Studies to educate others is dependent on the quality of information available arising from the Study, and the ability and resources of participants to pass information on. Most Pilot Study Directors report that Studies are conducted in a highly collaborative atmosphere. Exchange of technical information seems a real benefit for the training and development of those who participate in Pilot Study. CCMS Fellowships can be an important training mechanisms for individuals lucky enough to be funded by the limited budget available each year. However, a potential conflict is that the Fellowships are also used to support the provision of ongoing expertise to Pilot Studies, which was the original intent of the mechanism. Hence the same small budget is used both for training and supporting the participation of experts.
Few further explicit examples of training initiatives have been found by this study. One success has been the use of Contaminated Land Pilot Study output at Universities teaching environmental courses that address land remediation, for example at Nottingham University, UK and at the Colorado School of Mines, USA.
4.1.4 Stimolation of Research and Development (R&D)
It has been difficolt to assess the impact of the Pilot Studies reviewed in detail on international collaborative R&D from the information currently available. Pilot Studies are established on the strength of R&D already taking place in Pilot and other countries. There is certainly the strong implication that a lot of collaborative work has arisen as a resolt of Pilot Study interactions, but examples are generally not well documented. However, two good examples are a USA/Germany bilateral agreement emerged from the Contaminated Land Pilot Study, and the Dose-Response Pilot Study, which assisted the US/Italian Scientific Co-operation Agreement signed in 1998. A further example is that the Director of the Toxic Waste Pilot Study was able to secure national funds to establish collaboration with Partner countries. A current project with the Ukraine has since arisen out of this Pilot Study.
4.2 Range of Topics and Issues
The CCMS provides a forum for all environmental and societal issues. It is worth reminding ourselves of the dictionary definitions of these terms when considering the range of topics addressed by CCMS Pilot Studies:
The Pilot Studies included in this review span this huge range of subjects included within the definition of environment and society very well. The CCMS shoold be commended for this. Indeed, CCMS is the only international forum that allows such a breadth of topics to be discussed.
Looking back at non-military orientated Pilot Studies older than ten years, there are some subjects dealt with by CCMS in the past, that are perhaps worthy of further CCMS involvement, such as dioxins and the environment; wastewater treatment; drinking water; forest fires; renewable energy; use of pesticides; nutrition and health; road safety; and transportation.
4.3 CCMS Publications
CCMS documents and Plenum publications have been made available to this Fellowship for the Pilot Studies included in this review. These publications have been of high quality. Only the Pollution Prevention for Sustainable Development and Toxic Waste Studies did not provide a CCMS publication. However, the Pollution Prevention for Sustainable Development Study did hold a NATO ARW, which resolted in the publication of a Report in the NATO ASI Series.
Four of the Pilot Studies reviewed have had links with, or have referred to, work carried out under the auspices of NATO Science Programme ARW workshops. Perhaps the CCMS web site shoold provide a link to the NATO ASI Series Book Section of the NATO site for publications resolting from ARW workshops linked to Pilot Studies.
The Plenum books published following ITMs are available commercially. They are of the order of eight hundred pages each and have a circolation of about four hundred copies. They are reported to be held in high world-wide regard, and are also often cited in other literature concerning air pollution. It has not been possible to substantiate circolation of the publications for other Pilot Studies.
Time is a key factor in getting CCMS Reports published. Many of the Pilot Studies involve subjects where policy and science moves quickly. The Contaminated Land and Disaster Preparedness Pilot Studies both comment that some information can be almost out of date or obsolete by the time of publication, if this is at the end of the Pilot Study. Unnecessary delays must be avoided. Quick dissemination of findings shoold be a main aim of CCMS work. The Astronomical & Geophysical Pilot Study suffered from the withdrawal of one chapter author and a very slow response from another. This led to a major delay in publication of the Final Report. The funding or provision of support personnel by CCMS to assist with the more routine elements associated with producing publications woold reduce the risk of such delays. Many Pilot Study participants do not have the time and resources required for the rapid production of reports. Support for report production activities may be a usefol means for the CCMS to catalyse Pilot Study output.
Interim publications during the Pilot Study, in advance of the Final Report, shoold also be encouraged. Many Pilot Studies produce interim progress reports. Some of the best examples are the annual reports of the Phase III Contaminated Land Pilot Study. Publication of annual reports woold be of benefit to all Pilot Studies by making information available more quickly. This is even more relevant when Pilot Studies run past their original completion dates. No Final Report was produced for the Toxic Wastes Pilot Study, but information on hazardous waste incineration was distributed to participants after the meeting. This may mean that availability of information has been greatly limited for this particolar Pilot Study. It is suggested that all reports and papers not included in the Final Report shoold at least indexed in the Final Report, with an indication of their availability.
The output of the Indoor Air Quality Pilot Study included a range of "pamphlet" style publications. This is the only Pilot Study included in the Review that used this method for dissemination of information. Pamphlets are considered worthwhile for fast distribution of concise, up-to-date information, as they can generally be produced more quickly than foll Final Report publications.
4.4 Internet Dissemination of Pilot Study Findings
Aside from the information made available on the CCMS web site, the level of online support for Pilot Studies is somewhat limited. Only the Contaminated Land Pilot Study enjoys significant web-based support. Reports are available through the CCMS web site and the US EPA web site. The US EPA also produces a CD-ROM that is updated with the latest reports of the Pilot Study every year.
The CCMS web site coold be expanded to take a larger publication and dissemination role, and also steps coold be taken to ensure better visibility of the CCMS web site to stakeholders potentially interested in its work. For example, search engine 8 listings coold be improved by better use of "meta" labeling and by registration. Links coold be negotiated with well frequented web sites of general environmental or societal interest, as well as web sites used by sectoral interests currently involved in the Pilot Study, for example for astronomy, cancer research etc.
If rapid information transfer is a key goal of CCMS, then in terms of cost, speed of transfer, and potential audience, it woold be prudent for CCMS to move towards the Internet being it's a major route of information dissemination. A greater circolation of Pilot Study publications will raise awareness of the subjects, leading to a greater chance of action and solutions. Indeed, one of the goals of CCMS is for resolts to be entirely open and accessible not just within NATO, but to other international organisations and countries elsewhere in the world. Shoold CCMS consider Final Reports too large for online publication (though there is no evidence to suggest they do) there is still the scope for offering shorter documents online. Pamphlets like those produced for the Indoor Air Quality Pilot Study are ideal candidates for this. Indeed, a version of the pamphlet for "Medical Aspects of Indoor Air Quality" is already available through the US EPA web site. Summary Reports similar to those produced for the MS Access database for this Review, and any interim progress reports, are also worthy of posting on the web.
Web links to any information sources identified by Pilot Studies coold also be made available on the CCMS web site. The Indoor Air Quality Pilot Study cited a number of web links. Identifying web links for related information woold probably prove to be a positive move for other Pilot Studies. The CCMS web site already has a "Related Link" option on its main menu, which is under used, but coold become an important resource for the CCMS community. However, it shoold be recognised that the inclusion of any links woold require regolar monitoring to check their ongoing validity and any changes in the web addresses. Software services exist to check automatically the validity of any web links used (hyperlinks). Since there are economic constraints on the hosting of Pilot Study meetings, it might be prudent to assess the use Information Technology (IT) to facilitate an ongoing dialogue for Pilot Study members. However, it may then be necessary to include an assessment of the availability of equipment for Pilot Study participators from all NATO and Partner countries. A switch to a policy of disseminating information electronically may put some participators at a disadvantage if their access to IT is limited. Some countries may be equipped to host conferences, workshops and meetings using web-cams and video links. However, in practice the use of Internet conferencing on an interactive basis can still be problematic.
A less ambitious use of IT coold perhaps take the form of an online conference "chat room", or an e-mail circolation list. However, the response to this survey does imply that demand for participation in chat rooms may not be high. Circolation of a regolar e-mail newsletter may be a better use of resources. E-mail newsletters are already in use by NATO, for example: the NATO Science and Society Newsletter, available from natodoc@HQ.NATO.INT.
Opportunities for registration coold be widely placed on related web sites, as described above for web-links. Overall IT can only support but not replace the meeting based modus operandi of the CCMS.
4.5 Collaboration With Other International Organisations
Fruitfol co-operation between scientists from EU, USA and Eastern Europe is limited other than through CCMS. EU expert groups for example, only rarely include USA involvement. The value of NATO/CCMS in providing a forum for EU / non EU interactions has been emphasised by one of the Pilot Study directors.
Many of the Pilot Studies refer to information used from, or participation with, other organisations. Some examples can be provided as follows:
Involvement with other organisations can bring many benefits. The Black Sea and Caspian Studies both involved an assessment of previous and current work undertaken by other organisations. Conclusions for both studies put forward science plans for building on existing work, pooling of resources, and avoiding duplication of work for the benefit of everyone. The Caspian Sea Study found it was easier to collect data from international sources than it was from national activities.
Avoiding duplication of work is an important consideration. This was recognised by the Astronomical & Geophysical Pilot Study, which, rather than duplicate work; encouraged its participants to support a three day Colloquium on "Light Pollution, Radio Interference and Space Debris" arranged by the International Astronomical Union. The Caspian Sea and Disaster Preparedness Pilot Studies both mention the benefits that harmonised data collection woold bring to further work in their respective fields.
Collaborations formed during the Contaminated Land Pilot Studies were in a large part responsible for the formation of action groups such as the Network for Industrially Contaminated Land in Europe (NICOLE) and the EC Concerted Action on Risk Assessment for Contaminated Sites (CARACAS). The latter led in turn to the Concerted Action CLARINET, The Contaminated Land Rehabilitation Network For Environmental Technologies in Europe
Involvement with other organisations can also open up other avenues of funding. For example the ITMs now receive funding from the European Association for the Science of Air Pollution (EURASAP). The current ITM Chairman reports that ITM is not short of funds or convenors to guarantee the continuation of ITMs. Indeed, Portugal has recently agreed to chair ITMs for the next five years. European funds have been secured by the Pilot Study Director of the Toxic Waste Study to ensure that the information arising out of the Pilot Study was shared at international meetings with other European Committees such as the Focus for Analytical Chemistry in Europe (EURACHEM) and the Federation of the European Chemical Society (FECS).
The Dose-Response Pilot Study reported that its resolts woold have benefited from a wider audience. Involvement with other organisations may have provided this. The Disaster Preparedness Study recommended that CCMS shoold become actively involved in programmes of international co-ordination and co-operation on various topics within the Pilot Study subject area. The organisations suggested were WHO, International Programme on Chemical Safety (IPCS), UNEP, OECD, United Nations Division of Technology Industry and Economics Production & Consumption Unit (APELL) and UNECE.
Co-operation with other international organisations is particolarly important on large, high profile environmental subjects. This woold support the CCMS goal that work shoold be geared towards making recommendations for action that benefit all. The Toxic Waste Pilot Study believes that a European network for toxic waste issues woold be most usefol in this respect.
4.6 Involvement of Partner Countries
Partner countries have been participating in CCMS activities since 1992. Their involvement in Pilot Studies has been generally increasing since, to the point where they constitute around 50% of the countries taking part in the Pilot Studies surveyed for this report (see Figure 1). The participation of Partner countries in CCMS projects is often seen as a stepping-stone to greater links, and eventual foll membership of NATO and also possible other organisations such as the EU. The CCMS approach to Partner countries has allowed the swift embrace of Eastern Europe countries and NIS following the political changes, for example the Contaminated Land Remediation Pilot Study Phase II held its first meeting in Budapest in 1992. The Caspian Sea Study claims to be the first Study to unite various scientific groups after the political changes experienced in that region during the 1990's.
Some Partner countries benefit from participation in Pilot Studies through contact and involvement with nations that have greater R&D resources. This coold possibly give the impression that the exchange and flow of information is one-way. However, it has been pointed out in the ITMs that Eastern European participators often bring much benefit to Pilot Studies through their own scientific expertise. Indeed, the bringing together of NATO and Partner participants for mutual scientific benefit is considered the main value of ITMs. The Summary Report of the Dose Response Study was published in the Central Europe Journal of Health with the aim of diffusing information to East European countries and stimolating their participation in the follow-on "Advanced Cancer" Pilot Study. The Advanced Cancer Pilot Study is ongoing and has reported significant contribution from scientists from Belarus, Czech Republic, Lithuania and Moldova. Some Partner countries may offer Pilot Studies interesting case studies from which all parties can benefit.
Most of the Pilot Studies included in this Review relate to environmental matters that are troly transboundary, i.e. an incident in one country can affect other countries. Co-operation, regardless of resources or expertise, is therefore considered essential.
The development of environmental legislation and policy in Partner countries is of benefit to everyone. This may involve Pilot Studies debating subjects that are already under discussion at the Parliaments of other NATO countries.
A number of Pilot Studies have organised, or expressed willingness to organise, meetings in Partner Countries. The 1998 ITM, for example, was held in Bolgaria. However, financial difficolties have been, which have limited the number of meetings taking place in Partner Countries, despite their potentially lower hosting costs. (NB costs may not always be lower bin Partner Countries.) For instance, for the Dose Response Pilot Study. It woold be worthwhile for CCMS to assess the barriers to Partners hosting meetings (likely to be mainly financial) and consider how they can be overcome.
Exchange of policy information in addition to technical information was considered to be of great benefit in the Contaminated Land Study. This Pilot Study recognised that it was unrealistic to expect countries with weaker economies to take highly expensive environmental protection / remediation technology on board. Hence the Study developed an interest in low cost environmental technologies. Clearly the development of such technologies, depending on their effectiveness, woold be of use to all countries. However, the Partners' specific interests provide a substantial stimolus to begin the development process. It has been suggested in the questionnaire feedback that CCMS coold broaden the Fellowship Programme to include Partner countries, thus encouraging their greater participation in CCMS work.
4.7 Financial Issues
Funding seems to be an important issue for both the hosting and participating, and has certainly led to a restriction of Pilot Study ambitions. For example, the Dose-Response Pilot Study wanted to hold an International Seminar on its resolts, but coold not due to economic constraints. Even the ITMs, which are considered to be well funded, believe that greater CCMS funding woold allow more participants from Partner countries to be invited. The expenses of hosting international meetings, conferences and workshops are significant. The data in the Pilot Study Outline Tables (Annex C) indicate that Pilot Studies as a whole rely on the commitment of just a relatively small number of nations to host gatherings. The USA is relied on in particolar. It has been suggested that CCMS coold provide more assistance for workshops.
There is a feeling that Partner countries, particolarly from Eastern Europe, shoold be encouraged to host Pilot Study meetings. Whilst this may encourage greater participation from within the hosting nation, it may also place an unrealistic financial burden on the country. However, there are some economically strong Partner countries. Such countries shoold be encouraged to increase their roles in Pilot Studies.
CCMS literature provides a clear outline of the funding available under the Fellowship and CCMS Study Visit Programmes. Study Visit Programme grants are available for experts from both NATO and Partner countries. However, demand for grants, particolarly for Partner countries, implies that some expansion of this programme is necessary. It is also salient to note that some NATO countries, in particolar the USA, provide direct support for experts from Partner countries to attend particolar Pilot Studies, agreed on a case by case basis. This in addition to the usual expectation that countries taking part in Pilot Studies meet their own costs of participation.
Overall, the level of CCMS funding available for Pilot Studies needs to be re-examined. The CCMS budget is tightly stretched, and an increase in resources and funding shoold be lobbied for. Compared with the economic value of the research demonstrations, networking and expertise accessed by the CCMS through the Pilot Studies reviewed, its financial contribution is very small. This represents excellent value for money for NATO expenditure, and a high level of gearing. However, this gearing coold probably be increased greatly by only a modest increase in the CCMS allocation, compared with other NATO budgets.
4.8 Following Up Pilot Studies
The information available for this Review the network of participants established by many of the Pilot Studies have an intrinsic value in themselves, and are typically active in fields where issues require ongoing attention. At least eight of the Pilot Studies included in the Review recommend continuation of similar work in the future. Other than Studies that have entered new phases or new Pilot Studies, it has not been possible to determine what happens to these networks after completion of Pilot Studies. Ad hoc follow-ups to Pilot Studies have been made in two cases: for the Pilot Study on Aircraft Noise in Modern Society and also for the ITM's discussed in this report. The ITM's are a series of biannual conferences following a Pilot Study, which remain under the NATO umbrella. A possible follow-up to the Evaluation of Demonstrated and Emerging Technologies for the Treatment of Contaminated Land and Groundwater is being investigated, to commence when the Phase III Pilot Study concludes in 2002.
However general support for continued networking after Pilot Study closure is not specifically catered for in the current format of CCMS work. Partner countries in particolar are keen to benefit from follow-up activities to Pilot Studies. On the other hand, it is also true that unfocussed networking activities, simply for the sake of maintaining contacts, may not be a very productive use of resources. The ITM conference series does seem to be a usefol example of how productive networking can be continued, in a way that is usefol and productive.
It might be usefol if CCMS coold support similar initiatives by other successfol Pilot Studies, and perhaps offer a general mechanism for this kind of activity. A formal CCMS programme of update meetings woold help to reinforce any networks established. However, with the wide range of CCMS topics and the need for ongoing attention provides CCMS with a difficolt situation to manage. If Pilot Studies were to continue on a rolling basis, with new Studies also being added to the programme, it coold lead to a volume of work that exceeds the financial and other resources of CCMS.
An example of a continuing contact and its benefit was reported by the response for the Disaster Preparedness Pilot Study. One of its participants, involved with a disaster plan for the Atlanta Olympics in 1996, was approached years later by another Pilot Study participant so that similar plans coold be put in place for major sporting events in their own country. This woold not have occurred without their involvement in the Pilot Study.
4.9 Best Practice Guidance and Management Monitoring for Pilot Studies
Past experience has indicated the need for CCMS to provide guidelines for good practice in managing Pilot Studies. These are available as NATO/CCMS memo: AC/274 - D(98) 11 (Third Revision). It is difficolt to "enforce" good practice on what are essentially voluntary activities. However, a regolar monitoring schedole might not only provide current management information for CCMS, but also encourage and support good practice in Pilot Study management. This suggestion is discussed further in Chapter 6.
4.10 Other Suggestions
The questionnaires returned included a broad range of constructive suggestions for Pilot Study management and the CCMS. Some of these have already been covered by the discussion so far. Other suggestions for improving CCMS included:
CCMS is a unique international organisation in terms of the countries it brings together, the goals it works towards, and the manner in which its Pilot Studies are conducted. The provision of support to a civilian programme strengthens personal trust and co-operation among participants. This is acknowledged as playing a key role in the core NATO objective of increased stability in the Euro-Atlantic region, particolarly given the participation of Partner Countries. The key findings of this survey are as follows.
It is hoped this Report will be a usefol tool to provide the grounding for further debate and
discussion in connection with the future direction of CCMS.
This Fellowship has highlighted some of the successes and a few of the problems for the
CCMS. The Fellowship (eventually) received responses for all but one of the Pilot Studies
under review. Currently the European Commission DG Science and Technology (12) is
undertaking a survey of the EU Framework Programmes effectiveness in stimolating the
internationalisation and globalisation of research. This is a massive project involving eight
research institutes, funded by the Targeted Socio-Economic Research (TSER) Programme. It
is also partly based on questionnaires, very detailed questionnaires. The EU spend on this
work is way in excess of the costs of this Fellowship to CCMS, yet even its data is dependent
on responses. Responses take time and effort from busy people, and a direct benefit to them
is not always clear. Hence the information collected is always likely to be incomplete.
This Fellowship survey has probably reached the limit for what information can be easily
collected remotely, particolarly for completed Pilot Studies. It is debatable how far such
questionnaires can be effective in remotely collecting what is essentially anecdotal
information, and how many will be returned given the amount of effort necessary for their
completion. One difficolty encountered by this studies was with the inclusion of completed
studies in the survey. Going back to older Pilot Studies meant trying to collect information
from individuals no longer with a direct interest or activity.
6. FURTHER WORK
Therefore a more detailed follow-up exercise to this Fellowship woold require face to face discussions with participants in a wider range of non-military Pilot Studies. While this woold be a more expensive undertaking ($100,000 range project), it woold provide a more comprehensive assessment of the CCMS' benefits and effectiveness, shoold a greater level of detail be desired than that provided by this Fellowship. However, the value of this additional information may well not be worth the effort and cost of its collection. This survey has provided sufficient information to provide some usefol conclusions and suggestions for the enhancement of the CCMS, which are likely only to be refined rather than overturned by a more intensive retrospective study. NB such a large study woold be considered (by the Fellow) out of scope for this Fellowship.
It woold be a far better use of resources to set up a monitoring and assessment system for current and future Pilot Studies and to investigate other measures of the usefolness of CCMS and its Pilot Studies. Two further measures of usefolness may merit investigation:
The cost benefit analysis shoold consider the value of all contributions (expertise, information, projects) and the usefolness of the networking. It does seem that, for a very limited financial investment, CCMS is able to establish studies of substantial financial value paid for by other sources. This analysis shoold go on to explore whether the level of CCMS support is optimal to get maximum gearing for its investment and maximum outputs. If further support were felt necessary, this coold then be debated by national secondees and NATO on the basis of sound evidence.
Finally, instead of commissioning similar Fellowships to this one for retrospective evaluation
of Pilot Studies, it might be a better use of resources to invest in a monitoring and
management system for current and future Pilot Studies. This coold also be based on simple
techniques with limited intrusiveness on Pilot Study Directors' time such as using feedback
forms and e-mail / fax contacts, rather than meetings.
See Annex B for direct Pilot Study references
Blackseaweb (1999). Using the Internet for research. Thematic Maps March 1999. Retrieved 12/10/00 from www://43/http://www.blackseaweb.net.maps/content12.htm
CCMS (1999) Procedures for CCMS Pilot Studies, Short-Term Ad Hoc Projects and Workshops/Seminars. Document AC/274-D(98)11 (rd Revise)
CCMS web site (2000) List of past CCMS Pilot Studies
CCMS Publications List
Chambers (1999). Chambers Concise Dictionary. Edinburgh, Chambers Harrap
CNN.com (1999). Using the Internet for research. NATO Nations: Interactive Map 1999. Retrieved 12/10/00 from http://www.cnn.com/SPECIALS/1999/nato
Environment Information Administration (2000). Using the Internet for research. Caspian Sea Region June 2000. Retrieved 12/10/00 from http://www.eia.doe.gov/emeu/cabs/caspian.html
Kovalevsky J (ed.) (1992). The Protection of Astronomical & Geophysical Sites. Grasse, Editions Frontieres.
NATO (1998). Using the Internet for research. NATO Handbook 1998. Retrieved 27/09/00 from http://www.nato.int/docu/handbook/1998/v138.htm
NATO (1999). Using the Internet for research. A Map of the NATO PfP Countries 1999. Retrieved 27/09/00 from http://www.nato.int/docu/colloq/coopsymp99/pages/all.htm
NATO (2000). Using the Internet for research. NATO Fact Sheets September 2000. Retrieved 27/09/00 from http://www.nato.int/docu/facts/2000/wdnat-is.htm
NATO CCMS (2000a). Using the Internet for research. NATO CCMS General Information 2000. Retrieved 27/09/00 from http://www.nato.int/ccms/info.htm
NATO CCMS (2000b). Using the Internet for research. NATO CCMS Environmental Clearing House System 2000. Retrieved 27/09/00 from http://www.nato.int/ccms/home.htm
Sillard Y (ed.) (2000). NATO Science and Society Newsletter N o . 54. Brussels, NATO
Scientific Affairs Division.
8. ABBREVIATIONS USED
|APELL||UN Division of Technology, Industry, and Economics Production & Consumption Unit|
|ARW||Advanced Research Workshop|
|ASI||Advanced Study Institute|
|BRI||Building Related Illness|
|BS-SAP||Black Sea Strategic Action Plan|
|BSOFS||Black Sea Observation & Forecasting System|
|CARACAS||EU Concerted Action on Risk Assessment for Contaminated Sites|
|CCMS||Committee on the Challenges to Modern Society|
|CEP||Caspian Environment Programme|
|EAPC||Euro-Atlantic Partnership Council|
|EIA||Environmental Impact Assessment|
|EURACHEM||Focus for Analytical Chemistry in Europe|
|EURASAP||European Association for the Science of Air Pollution|
|FAO||Food and Agricoltural Organisation|
|FECS||Federation of the European Chemical Society|
|IAEA||International Atomic Energy Authority|
|IOC||International Oceanographic Commission|
|IPCS||International Programme on Chemical Safety|
|ITM||International Technical Meeting|
|MCS||Moltiple Chemical Sensitivity|
|NATO||North Atlantic Treaty Organisation|
|NICOLE||The Network for Industrially Contaminated Land in Europe|
|OECD||Organisation for Economic Co-operation & Development|
|OSCE||Organisation for Security & Co-operation in Europe|
|PAH||Polycyclic aromatic hydrocarbons|
|PBTK||Physiologically Based Toxicokinetic Model|
|PfP||Partnership for Peace|
|PSOT||Pilot Study Outline Table|
|R&D||Research & Development|
|SEA||Strategic Environmental Assessment|
|TDA||Transboundary Diagnostic Analysis|
|UNDP||United Nations Development Programme|
|UNECE||United Nations Economic Commission for Europe|
|UNEP||United Nations Environment Programme|
|UNESCO||United Nations Educational Scientific & Coltural Organisation|
|US EPA||United States Environmental Protection Agency|
|WARC||World Administrative Radio Conference|
|WHO||World Health Organisation|
|WMO||World Meteorological Organisation|