Figures
Abstract
The need to minimise consumer risk, especially for food that can be consumed uncooked, is a continuing public health concern, particularly in places where safe sanitation and hygienic practices are absent. The use of wastewater in agriculture has been associated with disease risks, though its relative significance in disease transmission remains unclear. This study aimed at identifying key risk factors for produce contamination at different entry points of the food chain. Over 500 produce and ready-to-eat salad samples were collected from fields, markets, and kitchens during the dry and wet seasons in Accra, Ghana, and over 300 soil and irrigation water samples were collected. All samples were analysed for E. coli, human adenovirus and norovirus using standard microbiological procedures, and real time RT-PCR. Finally, critical exposures associated with microbial quality of produce were assessed through observations and interviews. The study found that over 80% of produce samples were contaminated with E. coli, with median concentrations ranging from 0.64 to 3.84 Log E. coli/g produce. Prepared salad from street food vendors was found to be the most contaminated (4.23 Log E. coli/g), and that consumption of salad exceeded acceptable health limits. Key risk factors identified for produce contamination were irrigation water and soil at the farm level. Storage duration and temperature of produce had a significant influence on the quality of produce sold at markets, while observations revealed that the washed water used to rinse produce before sale was dirty. The source of produce and operating with a hygiene permit were found to influence salad microbial quality at kitchens. This study argues for a need to manage produce risk factors at all domains along the food chain, though it would be more effective to prioritise at markets and kitchens due to cost, ease of implementation and public health significance.
Citation: Antwi-Agyei P, Cairncross S, Peasey A, Price V, Bruce J, Baker K, et al. (2015) A Farm to Fork Risk Assessment for the Use of Wastewater in Agriculture in Accra, Ghana. PLoS ONE 10(11): e0142346. https://doi.org/10.1371/journal.pone.0142346
Editor: A. Mark Ibekwe, U. S. Salinity Lab, UNITED STATES
Received: August 19, 2015; Accepted: October 20, 2015; Published: November 10, 2015
Copyright: © 2015 Antwi-Agyei et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The study was funded by the Sanitation and Hygiene Applied Research for Equity (SHARE) Research consortium (www.shareresearch.org) through a UK aid from the Department for International Development (DFID). PAA received this funding for this study as part of his PhD scholarship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Although the full extent of disease burden attributable to food-borne diseases is largely unknown, food hygiene and food safety are major public health concerns. In 2005, for example, the World Health Organisation (WHO) attributed 1.8 million diarrhoea-related deaths largely to contaminated food and drinking water [1]. In the United States alone, an estimated 9.4 million episodes of food-borne illness, with 55,961 hospitalizations and 1,351 deaths are recorded every year [2]. Food-borne diseases result not only from consuming food contaminated with pathogens such as bacteria, viruses and parasites, but also chemicals or bio-toxins [2, 3]. The most often reported microbial agents related to food-borne diseases are Salmonella spp, norovirus, E. coli, Clostridium perfringens and Campylobacter spp [1, 2].
The risk factors for produce contamination are diverse, and may include environmental, animal and human sources. The use of urban wastewater for irrigation, post-harvest practices including market handling, and poor hygienic practices at kitchens have all been linked to produce contamination and disease outbreaks [4]. Although the health risks arising from urban wastewater use in agriculture are well documented, how consumer risk changes from field, to market, and to household are unclear and poorly documented. In most past studies, there tends to be a focus on disease risks analysis at the farm domain with very little at the market and kitchen domains. This lack of systematic assessment of food hygiene and safety along the complete food chain was the main thrust for the development of the Hazard Analysis and Critical Control Points (HACCP) in the food industry [5]. The HACCP helps identify critical pathways along the food chain where interventions could be prioritised; hence, specific risk-based targets can be developed to control hazards at the different steps in the food production chain [5]. This study adopted a HACCP approach to identify key risk factors associated with the microbial quality of produce from farm to fork.
Materials and Methods
Study area and site selection
The study was conducted in Accra, the capital city of Ghana with a population of 1.9 million [6]. Seven major agricultural sites were identified in the city where poor-quality water was used for the cultivation of salad vegetables, including lettuce, spring onions, cabbage, and local vegetables. Most crops are irrigated through the use of watering cans. Farmers sell their produce mainly to market vendors, but also directly to restaurants and street food vendors. Markets in Accra are classified into five types: central markets, neighbourhood markets, night markets, specialist markets, and privately managed markets [7]. The central markets serve as the largest platforms for vegetable sales, attracting traders from both within and outside Ghana. In Accra, the street food sector constitutes one of the biggest informal categories within the food industry. A popular category of street food vendor in Accra is the “check-check” seller. These are vendors who mostly sell cooked, or fried rice with salad (fast-food). Salads are normally prepared from lettuce, cabbage, carrots or spring onions, and can be mixed with or, without salad cream.
The three largest wastewater irrigated sites in Accra were selected for this study—Korle Bu, Dzorwulu and Marine Drive. Only farmers with at least one bed of ready-to-harvest lettuce at the time of study were included. Farmers were randomly selected using their farm beds/plot as identification. Three central markets (Makola, Agbobloshie and Kaneshie) were also included in the study. Vendors who were thought to sell both cabbage and lettuce were included in the study, and were randomly selected using their market stalls as identification. ‘Check-check’ vendors were recruited from a list of food vendors previously identified by a transect walk in two neighbourhoods (Old Fadama and Alajo) in Accra. Restaurants (including hotels) in Accra where salad was served to the public were also included in the study during the rainy season, and were selected on the basis of their “star” rating, location and popularity from the database of restaurants and hotels from the Ghana Food and Drugs Authority (FDA).
Data Collection
Sample collection and analysis.
Lettuce, soil and irrigation water samples were collected from wastewater irrigated fields, while lettuce and cabbage were collected from local markets. Sample collection at farms and markets was done between 7:00 hrs and 10:00 hrs and between 18:00 hrs and 21:00 hrs at street vending sites, all at peak working periods. Ready-to-eat salad samples from restaurants were collected between 10:00 hrs and 15:00 hrs. All samples were collected from September to December 2012 in the dry season, and from July to August 2013 in the rainy season.
At farms and markets, farmers and vendors were asked to place produce directly into plastic sampling bags (Whirl-Pak, USA) after they had cut off any roots to prevent unrelated contamination. The temperature of produce at markets was taken just before sample collection using a hand-held meter (ETI 226–010 ThermaLite, ETI Ltd, UK). For prepared food, vendors were asked to place the food sample into the opened sampling bag using whatever means (e.g. hands, utensils) but a note was made on how the food was handled. The presence of flies, the distance to open drains, refuse, and defaecation areas were also recorded, while observations were made on how the produce was displayed during sample collection. All collected samples were placed in an ice-box, and transported to the laboratory within 2 hours of collection for immediate processing, or stored in a refrigerator at 4°C until processing. At the laboratory 500 ml of sterile PBS (phosphate buffered saline, pH 7.2) was added to the bags, which were then vigorously shaken, and the surface of each piece of produce gently massaged through the bag before being processed, and analysed for E. coli, human adenovirus and norovirus genomes I and II. A 10 g of ready-to-eat salad sample was measured into a sterile tube, vortexed and shaken vigorously at room temperature before the supernatant was processed for the E. coli, norovirus and adenovirus assays. For farm soil, 10 g of the sample was measured into a sterile tube and 20 ml of sterile PBS added to it before 10 ml of supernatant was used for the assays.
All samples were processed using the membrane filtration technique with BBL MI agar (Beckton Dickinson, Sparks, USA) to determine the prevalence and concentrations of E. coli [8]. Serial dilution ranges were pre-optimized to ensure that ranges allowed enumeration of roughly 95% of samples, per sample type. RNA was extracted using the Qiagen Viral extraction kit (Qiagen, Venlo, Netherlands), and DNA using the MPBio FastDNA kit for Soil (MP Biomedicals, Santa Ana, USA). Virus presence/absence and inhibition in water, soil and produce/prepared salad was determined using Quantifast Pathogen IC Real Time—Polymerase Chain Reaction (RT-PCR) and PCR kits. Norovirus GI and GII and adenovirus concentrations were determined using Qiagen OneStep kits [9].
Observations.
Site observations were conducted in both seasons using structured observation guide, while participants’ observation was done in the dry season. Each farmer at wastewater irrigated fields and each vendor at markets was observed for one 3 hours session from 7:00 hrs to 10:00 hrs, whereas each street food vendor was observed from 18:00 hrs to 21:00 hrs. Farmers were observed during their farming activities including method of irrigation, application of poultry manure, and harvesting of produce. Market and street food vendors were observed on where and how they displayed and stored their produce, and any methods of treating produce/salad. In addition, general sanitation, including refuse, open drains, visible faeces, defecation areas as well as the presence of flies were observed. Participants were told that the observations were aimed at learning more about their general activities at farms, markets and street food vending sites, and not specifically to document critical health risk behaviours.
Questionnaire.
At the end of each participant observation, a standardised questionnaire was verbally administered to farmers, market salespersons, street food vendors, as well as chefs at restaurants. Questionnaires explored the sources, and the methods of displaying produce at markets and ready-to-eat salad at kitchens. It also covered where vendors sold and how produce was stored. At kitchens, the method of treating salad leaves as well as when the salad was prepared were recorded. In addition, the personal characteristics of participants including age, sex, religion, occupation and education were recorded.
Sample size
Sample size for produce was determined based on 80% power and 5% significance level to detect a 5% to 10% difference in faecal coliform concentration levels between produce at farms and markets [10]. This resulted in a sample size of 80 produce samples each from farms and markets during each of the dry and wet seasons. The number of soil and irrigation water samples collected at farms was assumed to correspond to the produce samples collected at farms during each season. Thirty samples of ready-to-eat salad were collected from 30 fast-food sellers (in each season) and 20 samples from chefs from 20 hotels and restaurants.
Data Analysis
All data were analysed using STATA 12 (StataCorp LP, College Station, USA). All samples with undetectable concentrations were multiplied by 0.5, the lower limit of detection for E. coli dilutions, or per standard curve in molecular virology analysis. Distributions of E. coli concentrations in environmental and food samples were tested for normality using the Shapiro-Wilk test. Concentrations were log transformed for calculations of means, standard deviations, and 95% confidence intervals (CI). The Mann-Whitney test was used to test for the difference in median concentrations of E. coli of street vended salad and irrigation water between seasons, while the Kruskal-Wallis test was used to compare the median concentrations of produce samples among the different domains. Apart from street-vended salad, two sample t-tests were used to compare the mean E. coli concentrations of produce at farms, markets and restaurants between the dry and rainy seasons as their distributions were normal after log transformation. Linear and logistic multiple regression models were used to assess risk factors for produce quality. Using a forward stepwise-regression approach, only risk factors that were significantly associated with produce quality at 20% were included in the multiple regression model [11]. Prior information, significance test and the size of effect-measure modification were used to determine the inclusion of interaction terms in the model. Multicollinearity was assessed using the variance inflation factor [12]. Statistically significant associations in the multivariable analysis were measured at 5% significance level using the likelihood ratio test.
Microbial concentrations in irrigation water were reclassified as ≤ 3 Log E. coli/100 ml and > 3 Log E. coli/100 ml following the (old) 1989 WHO water quality standard set for wastewater use [13]. Concentrations in salad produce were also regrouped as ≤ 3 Log E. coli/g and > 3 Log E. coli/g [14] or ≤ 2 Log E. coli/g and > 2 Log E. coli/g which define guidelines limits considered as microbiologically satisfactory for consumption [15, 16]. The proportion of produce and prepared salad with E. coli concentrations that met these guideline limits were then noted.
Quantitative Microbial Risk Assessment (QMRA)
In order to determine whether the consumption of wastewater irrigated produce met health standards, a quantitative microbial risk assessment (QMRA) model developed for the WHO guidelines for safe use of wastewater in agriculture was used [17]. The model uses the Karavarsamis-Hamilton method [18], together with the norovirus dose-response model developed by Teunis et al. [19]. A maximum tolerable additional disease burden of 10-6 disability-adjusted life years (DALYs) loss per person per year (pppy) as used in the WHO guidelines was adopted, which equates to a maximum permissible norovirus (NV) infection risk of 1.4 x 10-3 pppy.
The frequency and quantity of salad consumption were determined from questionnaire-based consumer surveys and laboratory experiments [20]. The amount of salad consumed on a daily basis at home was estimated using the average weight of lettuce bought at markets, and the number of lettuce used to prepare a salad meal for a family of 4 [20]. The quantity of salad consumed at street food vendor level was based on a national consumer survey by the International Water Management Institute in Ghana [21].
Five different consumption exposure, or pathway models were used to estimate the dose of norovirus ingested and subsequently, the risk of infection. The irrigation water model used the quality of irrigation water to estimate pathogen dose ingested based on the amount of wastewater left on produce after irrigation. All other models (farm produce model, market produce model, restaurant salad model and street food model) used direct E. coli concentrations on produce, or in prepared salad to estimate the dose ingested. A maximum pathogen reduction of 2 Log units arising from produce washing or disinfection was assumed for farm and market produce models [22]. No pathogen reduction was assumed for prepared salad models.
Ethical Considerations
Ethical approval was received from the London School of Hygiene and Tropical Medicine (reference number—6236) and from the Noguchi Memorial Institute of Medical Research, Accra, Ghana (Reference number—DF22). The study was explained to and agreed by local leaders, and written informed consent was obtained from each individual study participant and counter-signed and dated by researchers. All participants in this study were adults and were assured of confidentiality and security of the information they provide. Participants could only be identified by the use of alphanumeric symbols. The London School of Hygiene and Tropical Medicine granted permission for fieldwork. Access to hotels and restaurants was granted by the Ghana Food and Drugs Authority (FDA), while local government leaders (Assemblymen) approved of access to street food vendors in their communities. Access to markets and urban agriculture fields were also granted by the Secretaries and Market Queens of the market associations and the leaders of the farmer associations respectively. The field work did not involve any endangered or protective species.
Results
Microbiological quality of produce
A total of 422 produce samples were collected, 159 from wastewater irrigated fields and 263 from markets (134 lettuce & 129 cabbages), and a further 79 samples of ready-to-eat salads (59 from street food vendors and 20 from restaurants). Ready-to-eat salad from street food vendors was found to be the most contaminated, with 98% of all collected samples positive for E. coli, followed by market lettuce (97%), farm lettuce, (96%), market cabbage, (89%) and restaurants salads (80%). Overall, street salad was found to have the highest concentrations of E. coli (4.1 Log E. coli/g) among all produce and prepared salad samples (Fig 1). Farm lettuce was found to contain significantly higher levels of contamination than market lettuce for the combined seasons (3.3 vs 2.9 Log E. coli/g, p < 0.001). The concentrations of E. coli on farm lettuce during the dry season were found to be higher than during the rainy season (Table 1), in contrast to lettuce and cabbage at markets, which were found to contain higher concentrations during the rainy season (Table 2).
Solid horizontal line: limit of E. coli concentration classified as microbiologically satisfactory for consumption (% exceeding 2 Log E. coli/g—street vended salad, 90% (N = 59), farm lettuce, 88% (N = 159), market lettuce, 80% (N = 134), restaurants salad, 60% (N = 20), market cabbage, 18% (N = 129)). P-value calculated using Kruskal-Wallis test.
None of the produce samples from farms and markets were found to be positive for norovirus GI and GII, while 9% (N = 57) of farm produce, and 7% (N = 85) of market produce samples were positive for adenovirus. Mean concentrations of adenovirus in farm and market produce that tested positive for the virus were 8.1 x 103 and 1.9 x 104 gene copies/produce, respectively. No street vended salad sample was found positive for either one of the viruses.
Key exposures and practices associated with produce quality
The use of poultry manure as soil fertilizer was common and was higher in the dry season than in the rainy season (99% vs. 60%). The study also found that open defecation was common among farmers (73%), though the practice normally occurred away from the main farming areas. Although 68% of market vendors reportedly washed their vegetables (lettuce and carrots) before sales, observation of vendors’ washing practices at markets showed that washed water for produce was used without changing it for an average of 22 minutes, and the washed water was always dirty. At markets, at least 80% of produce were sold within 24 hours, but in some cases could be stored for 48 hours for lettuce, and 84 hours for cabbage before sale. At the street food sites, vendors used either public toilets (73%), or market toilets (27%). Generally, environmental sanitation at most street food sites was poor, with 87% of the sites without concrete or cement floors (Table 3). The 3-hour observations revealed that 33% of street food vendors had their salads uncovered at the time of sampling, and that salad could remain uncovered for an average time close to 100 minutes. Four of the six vendors who were observed to prepare salad at their vending sites, did not wash their hands before salad preparation. At vending sites, produce was stored for an average time of 10 hours before being used or sold.
Risk factors for produce microbial quality
The concentrations of E. coli found on farm produce increased with increased levels of E. coli found in soil or irrigation water (Fig 2). Seasonality modified the association between farm soil and farm produce quality with lower concentrations of E. coli found in the dry season as compared to the rainy season, with a 0.05 Log E. coli/g and 0.70 Log E. coli/g increase in produce contamination found per unit (Log E. coli/g) increase in soil contamination for the dry and rainy season respectively. In contrast, the effect of irrigation water quality on produce quality was found to be higher in the dry season as compared to the rainy season with a 0.20 Log E. coli/g and 0.06 Log E. coli/g increase in produce contamination per unit (Log E. coli/100 ml) increase of E. coli in irrigation water. However, the association between irrigation water and farm produce quality and the modification by seasonality was found to be non-significant (p = 0.19). The time of application of irrigation water, or poultry manure before sampling was found not to play any significant role on the concentration of E. coli found on farm produce (Table 1).
Seasonality and soil interaction p = 0.004, 95% CI = -0.85, -0.17. Soil effect on produce contamination (point estimate for unit increase = 0.60 Log E. coli/g produce, 95% CI = 0.32, 0.87, p < 0.001). Effect of irrigation water on produce contamination (point estimate for unit increase = 0.14 Log E. coli/g produce, 95% CI = 0.02, 0.27, p = 0.027). Error bars = 95% confidence intervals (CI).
No environmental exposures were identified at the markets that influenced the concentrations of E. coli found on market produce. However, every 1 hour increase in the storage time of lettuce was significantly associated with a 0.03 Log E. coli/g increased levels of produce quality (p = 0.05). On average, lettuce was stored for 10 hours at markets but for a maximum of 48 hours before sales. During the rainy season, a 1°C increase in the storage temperature of cabbage resulted in a reduction of 0.28 Log E. coli/g in the produce quality.
The method reported to decontaminate salad had a non-significant impact on the quality of street salad (Table 3). At hotels and restaurants, those who operated with a valid hygiene permit had on average 1.53 Log E. coli/g less contamination of their salad than those who had no valid hygiene permit (Table 4). The source of raw produce was also associated with a borderline significant difference in the average E. coli levels in salad sold at restaurants with the highest contamination or impact resulting from produce bought directly from farm gates (Table 4).
Produce quality and infection risk associated with salad consumption
On average, consumers of street food consumed 13 g of salad 4 times/week, while domestic consumers consumed 51 g of salad 2 times/week [20]. The median norovirus infection risk for the consumption of 10–51 g of lettuce salad for 2–4 days per week varied across the different exposure models and ranged between 2.6 x 10-3 and 0.32 pppy, and was highest with the street salad model (Table 5). The estimated infection risks from the irrigation water model, restaurant, and street salad models all exceeded the WHO guidelines. Only the risks from the consumption of produce of average contamination levels at farms and markets were marginally within the acceptable norovirus infection risks.
Discussion
The results of the study showed that salad produce was faecally contaminated at all entry points of the food chain, with street salad being the most contaminated. Key risk factors identified included farm soil contamination, and the use of wastewater for irrigation. Others were poor food and environmental hygiene, produce storage time and temperature, and operating without a hygiene permit. Based on the WHO QMRA model, the consumption of salads in Accra exceeded permissible health risk standards [23].
Produce quality from farm to fork
This study found street food salad to be the most faecally contaminated food, though concentrations of E. coli found at all sample points were high. The concentrations of faecal pathogens found on lettuce at farms in this study (3.31 Log E. coli/g) were lower than those found in previous studies in Ghana, ranging from 5.0 Log MPN/g to 9.0 Log MPN/g [24, 25], though most studies reported contamination in concentrations of total thermotolerant coliform (TTC), and not E. coli specifically [26]. However, the E. coli concentrations found on lettuce collected from agricultural fields in Accra were significantly higher than those in Pakistan [27] (>2,000 E. coli/g vs 1.9 E. coli/g produce), even though the irrigation water quality was found to be much better in Ghana (9.8 x 104 vs 1.8 x 107 E. coli/100 ml). This could most likely be explained by the type of irrigation water application, watering cans in this study, as compared to basin or furrow irrigation techniques, which minimize contact with wastewater; while the much higher temperatures and lower humidity in Pakistan could have promoted much more rapid die-off of E. coli on produce [27].
The microbial quality of produce is influenced by a variety of factors, which include: the type of vegetable, environmental conditions like temperature, humidity and exposure to sunlight, the type and the application of irrigation water, and post-harvest handling. Studies have shown that leafy vegetables with irregular surfaces tend to be more contaminated than smooth surface vegetables like cabbages [28, 29], and this could explain the higher levels of E. coli on market lettuce than cabbage in this study. A similar trend was also found by Amoah [24] in three cities (Accra, Kumasi and Tamale) in Ghana, where lettuce was found to harbour higher levels of faecal coliform than cabbage (1.1 x 107/g vs. 3.3 x 106/g).
The concentrations of E. coli in street food salad in this study (3.7 Log E. coli/g), although high, were lower than levels found in earlier studies in Accra, which were found to range between 5.1 and 6.4 Log cfu/g [30–34], though Mensah et al. [34] measured faecal coliforms. The higher levels of faecal contamination in those studies could be attributed to the fact that the salad were mixed with salad cream containing egg yolks, which has shown to be a good medium for microbial growth [30, 35]. Salad samples collected for this study were without salad cream. Most previous studies have also associated the contamination of street salad or raw produce to the sources of the produce, transportation practices, storage practices, method of salad preparation, and poor sanitation [30, 33, 36]. Similarly, the contamination of street food salad in this study could be associated with poor sanitation and hygiene practices at the vending sites. A study in Kumasi found higher levels of the TTC on produce (1.8 Log units/100 g) in salad from vendors with dirty sites and poor food handling practices [37]. The cut or sliced nature of the prepared salad also facilitated the growth of microorganisms, or increased their persistence, unlike raw produce which are intact [35]. The higher contamination of street food salad than salad from restaurants also confirms the findings of another study in Ghana, which found higher concentrations of TTC in salad sold by street vendors than those sold at cafeterias (5.4 Log cfu/100 g vs 3.8 Log cfu/100 g) [37].
The higher contamination of produce at farms than at markets emphasises the debate on the relative importance of post-harvest effect, including poor sanitation and market handling, on the quality of produce. Results from this study agree with earlier findings from Ghana [38], but contrast with the findings in Pakistan, where produce collected at a local market was found to have seven times higher levels of E. coli than produce from farm gates [27]. Apart from differences in irrigation methods, and environmental conditions that contributed to rapid pathogen die-off between the Ghana and Pakistan studies, the current study also seems to suggest that the differences might partly be influenced by seasonality, since farm produce was more contaminated than market produce only in the dry season and not in the rainy season. The design of the current study did not permit a direct correlation of farm-level contamination to produce contamination at markets, because produce at farms were not followed to markets to monitor the contamination levels. The geometric mean levels of E. coli found on market lettuce in this study (339 E. coli/g and 1,318 E. coli/g) were 20 times higher than levels found at markets in Faisalabad, Pakistan (14.3 E. coli /g) [27]. The contamination at Faisalabad was attributed to unsanitary market conditions and handling practices, including the method of washing produce.
The prevalence of human adenovirus (HAV) on farm produce was the first time viruses have been isolated in farm lettuce at the study sites. A recent study at the same sites analysed for human adenovirus and norovirus in wastewater, but not on farm produce [39]. Apart from norovirus, HAV is a known cause for food-borne infections, and its presence on farm produce could be attributed to the direct use of wastewater and animal manure for vegetable cultivation [40]. Moreover, viruses can survive on harvested produce and can remain infectious for several days up to a period of 5 weeks, even during storage [41, 42], and therefore could pose public health concerns. The study did not analyse samples for helminths, though earlier studies in Ghana have found helminth eggs on farm and market produce, and also in prepared salad sold at street food kitchens and also at cafeterias [10, 37].
Health risks associated with produce quality
Despite the increasing recognition of international guidelines on food safety such as the Codex Alimentarius, there are still some variations on the acceptable level of microbial concentrations in ready-to-eat food at different countries. While the International Commission on Microbiological Specifications for Foods (ICMSF) recommends a limit of 1,000 E. coli/g produce, the United Kingdom has a threshold of < 20 E. coli/g as satisfactory for consumption, and ≥ 100 E. coli/g as unsatisfactory for ready-to-eat food (including fresh vegetables and mixed salad vegetables) at the point of sale, a standard also adopted by Canada, Australia, New Zeeland and Hong Kong [16]. In Ghana, the national microbiological reference value for ready-to-eat foods including salad is < 100 cfu/g, and is based on the 3-class attribute sampling plan (satisfactory, acceptable, and unsatisfactory) [15]. This is the minimum count of organisms per gram, or per ml, below which there would be no risk associated with safety of a food, or the maximum value beyond which a lot would be rejected. Based on the Ghana standards, only 11% and 20% of the lettuce collected from the wastewater fields, and local markets could be deemed safe, and 40% and 10% from restaurants and street food vendors respectively. Results from the WHO QMRA model also seem to suggest that, the use of wastewater, and post handling practices at markets and kitchens in Accra are unsafe since the estimated pathogen risks were higher than the recommended level of 1.4 x 10-3 pppy for norovirus. However, for a relaxed DALY loss of 10-4, as proposed by Mara [43], almost all the median annual norovirus risk arising from the use of average produce/salad contamination were within the norovirus acceptable limits (0.14 pppy), though all the worst case scenarios exceeded this limit. Although all exposure models in this study were approximations, the restaurant and street salad models represented the closest estimate of the risk to consumers since these models used fewer assumptions, and are also at the points of direct consumption. The irrigation water model is the least reliable, and should not be used in instances where direct concentrations of pathogens on produce, or in prepared salad are available. The main disadvantage of the irrigation water model is that it assumes that the use of wastewater is the only source of produce contamination, and does not account for other sources of contamination at farms, markets and kitchens, nor does it take into consideration die-off between field and fork, or produce washing procedures. Only the risks from the farm and market produce models were marginally within the acceptable risks, and this could be due to the inclusion of pathogen reduction measures prior to consumption in those models as described in the methods section. On the other hand, the risks from these models still exceeded the threshold limits if the highest level of produce contamination were considered.
All models were based on an indicator-pathogen ratios, and this is a serious limitation of the models, and can lead to spurious results due to the complexities in these relationships, and their poor correlation with actual pathogen concentrations. Moreover, there is inadequate evidence to support the widely used ratio of 1:105 between bacteria indicator organisms and viruses, as the results from other studies seem to suggest that this ratio could result in overestimation of the risk. For example, a recent study [39] in Ghana found an average of one norovirus GII to 103.2 E. coli from its quantifiable irrigation water samples, while a component of the current study [44] also found the ratio of means between norovirus and E. coli as 2.1 x 10-2 or 1:101.7 from irrigation water samples analysed for both E. coli and norovirus, which are all larger than the common ratios used in recent publications. The presence of E. coli is an indicator of faecal pollution, but not necessarily a good indicator of disease risk, though signals the possible presence of other pathogens, and hence the need to intervene appropriately since “absolute zero risk” does not exist when assessing microbial risk in food [45]. In order to protect consumer health, a combination of produce washing and disinfection, which have been shown to reduce up to 3-Log unit of pathogens including norovirus is recommended; together with good agricultural practices [22, 46, 47]. Wastewater treatment and crop restriction are key risk reduction measures, but are rarely implemented in low and middle income countries.
Risk factors, health protective measures and policy implications
Probably the most significant public health concern found in this study was the high levels of faecal contamination found in street food salads. This is of particular concern, as up to 800,000 people per day have been estimated to consume this food, and other salad related foods from food establishments in major cities in Ghana [21]. The results of the study did not identify specific risk factors for street vended salad, though the use of some cleaning methods seemed to have a protective effect. The time between salad preparation and consumption could be a potential risk factor, as studies have shown that though most sanitizing solutions are capable of reducing microbial concentrations following washing, epiphytic microorganisms can grow rapidly, reaching similar levels as before washing [48, 49]. A key recommendation for street food vendors, therefore, will be to prepare salad in small quantities based on customer inflow, in order to prevent contamination due to inadequate storage and inappropriate temperatures. Another recommendation would be to prevent the use of leftover salad mixing with freshly prepared salad which could be another source of cross contamination [30]. Salads sold at hotels and restaurants must be prepared upon customer request, or be refrigerated (below 5°C) until ready to be served. Generally, microbial growth is slowed down, or stopped at temperatures below 5°C or above 60°C, though some psychotrophic microbes (e.g. Listeria monocytogenes, E. coli) may still develop, or multiply below 5°C if storage time is too long [50–52].
The potential effect of farm soil, and irrigation water on salad contamination could not be assessed due to the limitation of the study design. Observations at farm sites showed that some street food vendors bought vegetables directly from farms, and sometimes washed their produce with irrigation water, a practice that has also been reported among other market vendors in other parts of Ghana [24]. Other possible sources of salad contamination could be the chopping board, cutting knives, and working surfaces used for food preparation at kitchens, especially if these devices were used for multiple purposes such as cutting of meat [53, 54]. Street food vendors’ practice of not covering salad properly in receptacles during sales could worsen the microbial load due to cross-contamination [35]. This study found lower levels of contamination when a hygiene certificate was in place suggesting that local authorities should require vendors to obtain one. In order for these to work, frequent hygiene inspection, and monitoring of food premises by food hygiene authorities should remain a key priority. This is particularly necessary since vendors’ knowledge, awareness, and attitudes on hygiene alone do not necessarily translate into good hygienic practices [55]. Aside from cooking of vegetables and thorough washing and disinfection, domestic consumers can also remove the outer parts of vegetables before salad preparation to reduce the potential risk of pathogens, since most pathogenic contaminations are exogenic [56].
Produce quality at market showed non-significant associations with hygiene and sanitation practices, which could be as a result of this study unable to measure other potential risk factors for produce quality at the markets. Irrespective of this limitation, two possible reasons could account for produce contamination at markets. At first, washing of produce, could introduce microbial contaminants if wash water was contaminated as was shown at markets in Portugal [57] and Pakistan [27]. A study in Bangladesh, found wash water used for fresh vegetables and fruits to have enteric bacteria concentrations ranging from 1.3 103 to 2.0 107 cfu/ml [58]. Vendors are advised to wash produce under running potable water, or use multiple batches of potable water in order to prevent produce recontamination, though this requires involvement of local authorities as many markets lack access to clean water. A second reason for produce contamination at market could be environmental contaminants, as observations showed that 90% of vending sites were made of concrete, and therefore reducing the risk of dust as a potential source of produce contamination [59]. This study could not assess the role of factors such as transportation practices along the distribution chain, and handling during storage which could contribute to produce contamination at markets.
At farm level, contaminated soil and wastewater were found to be the main risk factors for produce quality. The higher impact of soil contamination on farm produce quality in the rainy season than in the dry season could be due to the frequent splashes of soil on produce arising from rainfall, and has been suggested by others [24, 60]. The use of poultry manure did not show significant association with produce quality, though this could be due to the fact that data was only collected on reported manure use, within the last four weeks. Similar findings were reported during field trials in Accra [24, 38], where direct wastewater use was the major risk factor for produce contamination, with contaminated soil and poultry manure identified as other potential sources of contamination. In this field trial study, the use of manure increased the levels of faecal coliform on lettuce cultivated on average by tenfold, while the use of wastewater increased faecal coliform levels 0–100 fold. The high use of manure in this study was similar to those reported in previous studies (73% to 98%) in Accra and Kumasi, where manure was found to be highly contaminated with faecal coliforms ranging from 1.0 103 to 1.0 108/100 g [10, 38, 61]. Poultry manure is recommended only as soil amendment if it has been adequately dried, i.e. composted aerobically to levels between 60°C and 80°C, and for at least 15 days before application [51], something which was not reported to be practiced by farmers. Restrictions on the use of untreated wastewater, the adoption of crop restriction, together with drying of poultry manure remain the best ways to ensure food safety in Accra. However, these measures are difficult to implement in resource constrained settings as a result of high cost, lack of alternative sources of irrigation water, and farmers’ unwillingness to cultivate non-vegetables, or non-food crops for loss of profits. In countries where good quality irrigation water sources are unavailable in the short to medium term, interventions that require less restrictions and minimal financial investment from farmers are recommended, as farmers find these more attractive to adopt [62]. These interventions include agricultural practices that limit produce contact to sediments such as controlled fetching of irrigation water from sedimentation ponds, the use of watering cans fitted with caps or fabric filters to reduce splashes of contaminated soil on produce, and the use of simple filters [60, 62, 63]. These interventions, however, need to be tested widely to assess their implementation successes and challenges. Education on the proper use of manure, as well as local authorities’ collaboration and support to farmers to gain access to alternative fertilizers, and water sources such as wells are also significant measures that can be taken to mitigate exposure and health risks.
Conclusions
The results of this study suggest that the use of untreated wastewater poses significant risks for produce contamination at the farm level, but its role in influencing consumer risks at markets and kitchens remains unclear. Though produce was contaminated at all entry points of the food chain, street food was found to be the most contaminated, and therefore the most critical domain because it is at the actual point of consumption, and also due to the large number of people who consume street food, and the fact that it poses the greatest risk to public health as was supported by the QMRA models. The study recommends field trials, or related studies, to establish the effect of wastewater irrigation on produce quality and hence consumer risks at markets and kitchens, and whether this effect is significant in terms of public health. It also recommends an assessment on the influence of produce washing practices at markets, and vendors’ preparation, handling and management of salad during sales, on the quality of salad produce. Lastly, further studies are needed to determine the presence and concentrations of pathogens to help improve the quality of QMRA estimates. The study concludes that, in as much as interventions at the source of production (farms) may result in significant positive public health impact, adequate hygienic practices including hygiene inspection, hygiene certification and improvements in general environmental and food hygiene practices at markets, but especially at points of consumption (food kitchens) are regarded as more effective in terms of cost, ease of implementation and above all public health significance. At farm sites, access to credit schemes and improved land security are recommended measures to encourage farmers to adopt risk reduction measures.
Supporting Information
S1 Supporting Information.
Appendix A. Irrigation water sample collection form
Appendix B. Particulate (soil) sample collection form
Appendix C. Raw produce (farm) sample collection form
Appendix D. Raw produce (market) sample collection form
Appendix E. Food (salad) sample collection form
Appendix F. Questionnaire for farmers
Appendix G. Questionnaire for market vendors
Appendix H. Questionnaire for produce buyers (market)
Appendix I. Questionnaire for street food vendors
Appendix J. Questionnaire for street food consumers
Appendix K. Observation guide for farm workers
Appendix L. Observation guide for market vendors
Appendix M. Observation guide for street food vendors
https://doi.org/10.1371/journal.pone.0142346.s001
(PDF)
Acknowledgments
Special thanks go to Maud Amon Tanoh and Sheillah Simiyu for reviewing the draft manuscript. I am also grateful to Prof. Duncan Mara, Dr Andrew Sleigh and Dr Barbara Evans, all of the University of Leeds, UK, for their guidance on the use of the WHO QMRA model for wastewater reuse. I am also thankful to the Environmental Health Group (EHG) at the London School of Hygiene and Tropical Medicine for their suggestions to improve the manuscript. Lastly, I am indebted to all study participants and their leaders and staff of the SANIPATH project in Accra, Ghana for their support throughout the study.
Author Contributions
Conceived and designed the experiments: PAA JE KB CM JA GA AP. Performed the experiments: PAA KB CM JA GA VP. Analyzed the data: PAA JE JB. Contributed reagents/materials/analysis tools: PAA JE VP KB CM JA GA JB. Wrote the paper: PAA SC AP VP JB KB CM JA GA JE.
References
- 1. Newell DG, Koopmans M, Verhoef L, Duizer E, Aidara-Kane A, Sprong H, et al. Food-borne diseases—the challenges of 20years ago still persist while new ones continue to emerge. International journal of food microbiology. 2010;139:S3–S15. pmid:20153070
- 2. Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, et al. Foodborne illness acquired in the United States—major pathogens. Emerging infectious diseases. 2011;17(1).
- 3.
Pepper I, Gerba C, Gentry T. Environmental Microbiology. In: Pepper IL, Gerba CP, Gentry TJ, editors. Environmental Microbiology (Third edition). San Diego: Academic Press; 2015. p. 509–45.
- 4.
WHO. Guidelines for the safe use of wastewater, excreta and greywater: Wastewater use in agriculture (Volume II). Retrieved from persistent URL: http://www.who.int/water_sanitation_health/wastewater/gsuweg2/en/index.html. 2006.
- 5.
WHO. Food safety risk analysis: A guide for national food safety authorities. World Health Organization. 2006.
- 6.
GSS. 2010 Population and Housing census. Accra: Ghana Statistical Service (GSS). 2012.
- 7.
AMA. Medium Term Development Plan 2006–2009: Volume One Profile of Accra. 2007.
- 8.
Oshiro R. Method 1604: Total Coliforms and Escherichia coli in water by membrane filtration using a simultaneous detection technique (MI Medium). Washington, DC: US Environmental Protection Agency. 2002.
- 9. Liu P, Hsiao HM, Jaykus LA, Moe C. Quantification of Norwalk virus inocula: Comparison of endpoint titration and real-time reverse transcription-PCR methods. Journal of medical virology. 2010;82(9):1612–6. pmid:20648618
- 10. Amoah P, Drechsel P, Abaidoo R, Henseler M. Irrigated urban vegetable production in Ghana: Microbiological contamination in farms and markets and associated consumer risk groups. Journal of Water and Health. 2007;5(3):455–66. pmid:17878560
- 11.
Kirkwood BR, Sterne JAC. Essential medical statistics Blackwell publishing. 2005;(Second edition).
- 12.
Hamilton L. Statistics with Stata: version 12: Cengage Learning; 2012.
- 13.
WHO. Health guidelines for the use of wastewater in agriculture and aquaculture: Report of a WHO scientific group: World Health Organization; 1989.
- 14.
ICMSF. International Commission on Microbiological Specification for Food. Microorganisms in foods. Sampling for microbiological analysis: Principles and specific Applications. University of Toronto Press, Toronto. 1974.
- 15.
Ghana Standard Board (Authority) G. Ghana Standard Board (GS 955:2009) Microbiological Analysis of Foods—Sampling Plans and Microbiological Criteria. ICS 0.7.100.30; 67.050. 2009.
- 16. Gilbert R, De Louvois J, Donovan T, Little C, Nye K, Ribeiro C, et al. Guidelines for the microbiological quality of some ready-to-eat foods sampled at the point of sale. PHLS Advisory Committee for Food and Dairy Products. Communicable disease and public health/PHLS. 2000;3(3):163–7. pmid:11014026
- 17.
Mara D, Sleigh A. Quantitative Microbial Risk Analysis: The 2006 WHO Guidelines and Beyond− How to Use the QMRA-Monte Carlo Computer Programs in Wastewater-use Planning. School of Civil Engineering, University of Leeds, Leeds. 2010.
- 18. Karavarsamis N, Hamilton A. Estimators of annual probability of infection for quantitative microbial risk assessment. Journal of Water and Health. 2010;8(2):365–73. pmid:20154399
- 19. Teunis PFM, Moe CL, Liu P, E Miller S, Lindesmith L, Baric RS, et al. Norwalk virus: How infectious is it? Journal of medical virology. 2008;80(8):1468–76. pmid:18551613
- 20.
Antwi-Agyei P, Bruce J, Peasey A, Biran A, Ensink JHJ. Risk perceptions of wastewater use for urban agriculture in Accra, Ghana. In press.
- 21.
Drechsel P, Keraita B. Irrigated Urban Vegetable Production in Ghana: Characteristics, Benefits and Risk Mitigation: IWMI; 2014.
- 22. Amoah P, Drechsel P, Abaidoo R, Klutse A. Effectiveness of common and improved sanitary washing methods in selected cities of West Africa for the reduction of coliform bacteria and helminth eggs on vegetables. Tropical Medicine & International Health. 2007;12(s2):40–50.
- 23.
WHO. Guidelines for the safe use of wastewater, excreta and greywater: Wastewater use in agriculture (Volume II). Retrieved from persistent URL: http://www.who.int/water_sanitation_health/wastewater/gsuweg2/en/index.html; 2006.
- 24.
Amoah P. Wastewater Irrigated Vegetable Production:Contamination pathway for health risk reduction in Accra, Kumasi and Tamale—Ghana, PhD Thesis, Department of Theoretical and Applied Biology, Kwame Nkrumah University of Science and Technology, Kumasi—Ghana. 2008.
- 25. Keraita B, Konradsen F, Drechsel P, Abaidoo RC. Effect of low‐cost irrigation methods on microbial contamination of lettuce irrigated with untreated wastewater. Tropical Medicine & International Health. 2007;12(s2):15–22.
- 26.
Stevens M, Ashbolt N, Cunliffe D. Review of coliforms as microbial indicators of drinking water quality. NHMRC, Biotext Pty Ltd, Canberra. 2003.
- 27. Ensink JHJ, Mahmood T, Dalsgaard A. Wastewater-irrigated vegetables: market handling versus irrigation water quality. Tropical Medicine & International Health. 2007;12:2–7.
- 28. Parish M, Beuchat L, Suslow T, Harris L, Garrett E, Farber J, et al. Methods to Reduce/Eliminate Pathogens from Fresh and Fresh-Cut Produce. Comprehensive reviews in food science and food safety. 2003;2(s1):161–73.
- 29. Ölmez H, Kretzschmar U. Potential alternative disinfection methods for organic fresh-cut industry for minimizing water consumption and environmental impact. LWT-Food Science and Technology. 2009;42(3):686–93.
- 30. Ameko E, Achio S, Alhassan S, Kassim A. Microbial safety of raw mixed-vegetable salad sold as an accompaniment to street vended cooked rice in Accra, Ghana. African Journal of Biotechnology. 2012;11(50):11078–85.
- 31. Yeboah-Manu D, Kpeli G, Akyeh M, Bimi L. Bacteriological Quality of Ready-to-Eat Foods Sold on and Around University of Ghana Campus. Research Journal of Microbiology. 2010;5(2).
- 32. Adu-Gyamfi A, Nketsia-Tabiri J. Microbiological studies of macaroni and vegetable salads in Waakye, a local street food. Ghana Journal of Science. 2008;47(1):3–9.
- 33. Feglo P, Sakyi K. Bacterial contamination of street vending food in Kumasi, Ghana. Journal of Medical and Biomedical Sciences. 2012;1(1):1–8.
- 34. Mensah P, Yeboah-Manu D, Owusu-Darko K, Ablordey A. Street foods in Accra, Ghana: how safe are they? Bulletin of the World Health Organization. 2002;80(7):546–54. pmid:12163918
- 35. Harris L, Farber J, Beuchat L, Parish M, Suslow T, Garrett E, et al. Outbreaks Associated with Fresh Produce: Incidence, Growth, and Survival of Pathogens in Fresh and Fresh-Cut Produce. Comprehensive reviews in food science and food safety. 2003;2(s1):78–141.
- 36. Olaimat AN, Holley RA. Factors influencing the microbial safety of fresh produce: A review. Food microbiology. 2012;32(1):1–19. pmid:22850369
- 37. Fung J, Keraita B, Konradsen F, Moe C, Akple M. Microbiological quality of urban vended salad and its association with gastrointestinal diseases in Kumasi, Ghana. International Journal of Food Safety, Nutrition and Public Health. 2011;4(2):152–66.
- 38. Amoah P, Drechsel P, Abaidoo R. Irrigated urban vegetable production in Ghana: Sources of pathogen contamination and health risk elimination. Irrigation and Drainage. 2005;54(S1):S49–S61.
- 39. Silverman AI, Akrong MO, Amoah P, Drechsel P, Nelson KL. Quantification of human norovirus GII, human adenovirus, and fecal indicator organisms in wastewater used for irrigation in Accra, Ghana. Journal of Water & Health. 2013;11(3).
- 40. Rodriguez-Lazaro D, Cook N, Ruggeri FM, Sellwood J, Nasser A, Nascimento MSJ, et al. Virus hazards from food, water and other contaminated environments. FEMS microbiology reviews. 2012;36(4):786–814. pmid:22091646
- 41. Carter M. Enterically infecting viruses: pathogenicity, transmission and significance for food and waterborne infection. Journal of Applied Microbiology. 2005;98(6):1354–80. pmid:15916649
- 42.
Bosch A, Pintó RM, Abad FX. Survival and transport of enteric viruses in the environment. Viruses in foods: Springer; 2006. p. 151–87.
- 43.
Mara D, Hamilton A, Sleigh A, Karavarsamis N. Discussion paper: options for updating the 2006 WHO guidelines. WHO, FAO, IDRC, IWMI. 2010.
- 44.
Antwi-Agyei P, Ensink JHJ, Hunter P. A quantitative microbial risk assessment (QMRA) of norovirus risk to consumers of salad produce irrigated with wastewater. In press.
- 45. WHO, FAO. Risk characterization of Microbiological Hazards in Food—Guidelines. Microbiological Risk Assessment Series. 2009;17.
- 46. Shuval H, Lampert Y, Fattal B. Development of a risk assessment approach for evaluating wastewater reuse standards for agriculture. Water Science and Technology. 1997;35(11):15–20.
- 47.
Keraita B. ‘Low-cost measures for reducing health risks in wastewater irrigated urban vegetable farming in Ghana’, PhD thesis, Faculty of Health Sciences, University of Copenhagen. 2008.
- 48. Gil MI, Selma MV, López-Gálvez F, Allende A. Fresh-cut product sanitation and wash water disinfection: problems and solutions. International journal of food microbiology. 2009;134(1):37–45.
- 49. Allende A, Selma MV, López-Gálvez F, Villaescusa R, Gil MI. Role of commercial sanitizers and washing systems on epiphytic microorganisms and sensory quality of fresh-cut escarole and lettuce. Postharvest Biology and Technology. 2008;49(1):155–63.
- 50.
Barbarosa-Cánovas GV. Handling and preservation of fruits and vegetables by combined methods for rural areas: technical manual: Fao Inter-Departmental Working Group; 2003.
- 51.
Camelo AFL. Manual for the preparation and sale of fruits and vegetables: from field to market: Food & Agriculture Org.; 2004.
- 52. Brackett R. Incidence, contributing factors, and control of bacterial pathogens in produce. Postharvest Biology and Technology. 1999;15(3):305–11.
- 53.
Garbutt J. Essentials of food microbiology: Arnold, Hodder Headline plc; 1997.
- 54. Addo KK, Mensah GI, Bonsu C, Akyeh ML, Addo K. Food and its preparation conditions in hotels in Accra, Ghana: a concern for food safety. African Journal of Food Agriculture Nutrition and Development. 2007;7(5):1–12.
- 55. Annor GA, Baiden EA. Evaluation of Food Hygiene Knowledge Attitudes and Practices of Food Handlers in Food Businesses in Accra, Ghana. Food & Nutrition Sciences. 2011;2(8).
- 56. Minhas P, Sharma N, Yadav R, Joshi P. Prevalence and control of pathogenic contamination in some sewage irrigated vegetable, forage and cereal grain crops. Bioresource technology. 2006;97(10):1174–8. pmid:16023342
- 57. Vaz da Costa Vargas S, Bastos R, Mara D. Bacteriological aspects of wastewater irrigation. Tropical Public Health Engineering (TPHE) Research Monograph no. 1996;8.
- 58. Ali M, Khan M, Saha ML. Antibiotic resistant patterns of bacterial isolates from ready-to-eat (RTE) street vended fresh vegetables and fruits in Dhaka City. Bangladesh Journal of Scientific Research. 2012;24(2):127–34.
- 59.
Muinde O, Kuria E. Hygienic and sanitary practices of vendors of street foods in Nairobi, Kenya. 2005.
- 60. Keraita B, Drechsel P, Konradsen F, Vreugdenhil RC. Potential of simple filters to improve microbial quality of irrigation water used in urban vegetable farming in Ghana. Journal of Environmental Science and Health, Part A. 2008;43(7):749–55.
- 61. Drechsel P, Abaidoo R, Amoah P, Cofie O. Increasing use of poultry manure in and around Kumasi, Ghana: Is farmers’ race consumers’ fate. Urban Agricultural Magazine. 2000;2:25–7.
- 62. Keraita B, Drechsel P, Konradsen F. Perceptions of farmers on health risks and risk reduction measures in wastewater irrigated urban vegetable farming in Ghana. Journal of Risk Research. 2008;11(8):1047–61.
- 63. Keraita B, Drechel P, Konradsen F. Using on-farm sedimentation ponds to improve microbial quality of irrigation water in urban vegetable farming in Ghana. Water Science & Technology. 2008;57(4).