Report of the workshop on taxonomic quality issues in the DATRAS database (WKTQD)
MetadataShow full item record
Original versionThis report is not to be quoted without prior consultation with the General Secretary.
Quality control of data collections is an issue of primary importance in conducting science and this is no different for surveys intended to monitor changes in fish abundance. Essentially, quality control is the responsibility of the national institutes conducting the surveys. However, if the national data are combined in a common international data base, as is the case with DATRAS, the consistency of the data submitted engages an international dimension, because the reliability of any comprehensive analysis of changes in the fish community at large that could serve as the basis for ICES advice depends on the reliability of the species identifications in all national subsets. Many of the surveys that are routinely carried out jointly under the auspices of ICES have a long history during which the primary objectives have changed. Thus, the IBTS started as a Young Herring Survey, was then transformed in a Young Fish Survey to obtain recruitment estimates of commercial species, and became only a general monitoring survey of the entire fish community at a later stage. Despite these changes in general objectives, the emphasis in data use is still largely focused on the commercial species that are relatively easy to identify. Although measurement errors and punching errors for this group may have entered the data base, as evident from unrealistically small or large individuals reported in some cases, it seems generally safe to conclude that, in view of the large amount of detailed data collected for these species, these could lead only to minor and negligible distortions in the analyses. However, when it comes to the less common species, studies of the IBTS component in the past have proven major inconsistencies in species identification in the data set that has been entered in DATRAS. This problem is not restricted to a single or a few countries, but affects all countries, although the species involved may differ. This suggests that it is a direct consequence of the large number of people involved in data collection on board and of a generic lack of good taxonomic knowledge among the scientific staff at large that inhibits the maintenance of enough quality control. The problems identified in the past have been the direct reason for holding this one-off workshop to discuss the various aspects of identifying inconsistencies and correcting species identifications in historic data sets and of ensuring correct species identification in future data collections. Taxonomic quality control is a complex issue, because the problems vary by region depending on the species that may be encountered and therefore may require specific approaches regionally. Moreover, the ultimate responsibility for introducing specific protocols for quality control rests with the survey working groups responsible for data collection rather than that the appropriate procedures can be prescribed by others. Therefore, the aim of the workshop has been to provide generic guidelines for development of suitable protocols by the survey working groups rather than to come up with a final answer. It must be emphasized that so far progress in identifying inconsistencies in reporting of various taxa has been restricted to the IBTS component of DATRAS, which covers the North Sea, Skagerak and Kattegat. For all other surveys, similar analyses have not yet been conducted, but there is no reason to assume that the situation would be any different. Following the Terms of Reference, the report is split in four sections that deal with each of these respectively. Section 3 deals with ToR a): “Identify and correct taxonomic mis-identifications and input errors in DATRAS”. Obviously, this ultimate goal was beyond reach during a three-day workshop, and a lot more work needs to be done. This section lists dubious species, inconsistent information provided regarding taxonomic level reported, maximum attainable size and area of distribution, and examples of inconsistent information reported for some problematic taxa. The information given is restricted to the IBTS component of DATRAS, but should not be interpreted as a comprehensive analysis of all inconsistencies that may be 2 | ICES WKTQD Report 2007 present in this data set. Rather, it highlights methodical aspects as to how inconsistencies may be elucidated. Section 4 deals with ToR b): ”Development of protocols for ensuring the appropriate treatment of data reported at higher taxonomic levels”. Historically, different countries have reported variously species at different taxonomic levels (genus or families). Also, uncertain species records may have to be adjusted by using a higher taxonomic level. As a consequence, subsequent community analyses may require that these higher taxa are split into its constituting species based on identifications considered reliable. Because there are various ways to do such computations depending on the assumptions made, different analyses could give different answers. From an ICES perspective, some consistency in the approaches used by different working groups would seem appropriate. This section provides the essentials of an appropriate algorithm based on using length frequencies, area of distribution and year of catch, that could serve as a first guideline for comprehensive community analyses as well as for trend analysis of individual species. Section 5 deals with ToR c): “Develop improved protocols to ensure that species identification in trawl surveys is appropriate for fish community studies, including the development of photo-ID keys for nations participating in surveys”. Various initiatives have been taken by individual countries to develop appropriate tools for species identification, including training courses. This section lists a number of ways by which future data collections may be improved or by which species identification can be ascertained at a later stage in the process. Finally, section 6 deals with ToR d). “Develop protocols for (i) improving quality control during the submission of data to DATRAS and (ii) the future checking and quality assurance of DATRAS data. This ToR relates to the important aspect of the responsibility of ICES for ensuring that all taxonomic data in DATRAS are correct or, if they are dubious but cannot be corrected, that the information provided to external users is properly identified as being of a dubious nature. The section describes a warning system that should be developed at the submission stage of new data so as to inform each country that the data submitted contain information that is inconsistent with what is known about the biology of the species and therefore should be carefully checked. It is also stressed that historic data that cannot be corrected but remain dubious are properly flagged internally and potentially adjusted when made available to external users. Proper taxonomic quality control will remain an issue of all monitoring programmes that needs continuous attention and adjustments. We can only hope that this comprehensive description of its many aspects helps the survey working groups to make real progress in achieving an urgently required revision of the historic information provided as well as in improving future data submission.
Contributor: Franz Uiblein
SeriesICES CM documents