The JI
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     
 

The Journal of Immunology, 2007, 179, 8418 -8424
Copyright © 2007 by The American Association of Immunologists, Inc.
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow A correction has been published
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow Request Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ryan, A. A.
Right arrow Articles by Triccas, J. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ryan, A. A.
Right arrow Articles by Triccas, J. A.

Improved Protection against Disseminated Tuberculosis by Mycobacterium bovis Bacillus Calmette-Guérin Secreting Murine GM-CSF Is Associated with Expansion and Activation of APCs1

Anthony A. Ryan*, Teresa M. Wozniak*, Elena Shklovskaya*, Michael A. O’Donnell§, Barbara Fazekas de St. Groth*, Warwick J. Britton*,{dagger} and James A. Triccas2,*,{ddagger}

* Centenary Institute of Cancer Medicine and Cell Biology, {dagger} Discipline of Medicine, and {ddagger} Microbial Pathogenesis and Immunity Group, Discipline of Infectious Diseases, University of Sydney, Camperdown, Australia; and § Department of Urology, University of Iowa Hospitals and Clinics, Iowa City, IA 52240


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 Disclosures
 References
 
Modulating the host-immune response by the use of recombinant vaccines is a potential strategy to improve protection against microbial pathogens. In this study, we sought to determine whether secretion of murine GM-CSF by the bacillus Calmette-Guérin (BCG) vaccine influenced protective immunity against Mycobacterium tuberculosis. BCG-derived GM-CSF stimulated the in vitro generation of functional APCs from murine bone marrow precursors, as demonstrated by the infection-induced secretion of IL-12 by differentiated APCs, and the ability of these cells to present Ag to mycobacterium-specific T cells. Mice vaccinated with BCG-secreting murine GM-CSF (BCG:GM-CSF) showed increased numbers of CD11c+MHCII+ and CD11c–CD11b+F480+ cells compared with those vaccinated with control BCG, and this effect was most apparent in the draining lymph nodes at 7 and 14 days postvaccination. Vaccination with BCG:GM-CSF also resulted in enhanced expression of costimulatory molecules on migratory dendritic cells in the draining lymph nodes. The increased APC number was associated with an increase in the frequency of anti-mycobacterial IFN-{gamma}-secreting T cells generated after BCG:GM-CSF vaccination compared with vaccination with control BCG, and this effect was sustained up to 17 wk in the spleens of immunized mice. Vaccination with BCG:GM-CSF resulted in an ~10-fold increase in protection against disseminated M. tuberculosis infection compared with control BCG. This study demonstrates the potential of BCG-secreting immunostimulatory molecules as vaccines to protect against tuberculosis and suggests BCG:GM-CSF merits further appraisal as a candidate to control M. tuberculosis infection in humans.


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 Disclosures
 References
 
Anti-mycobacterial immunity is established in the lymph nodes that drain the site of infection, triggered by the presentation of mycobacterial Ags via the MHC class I and II pathways (1). Presentation of mycobacterial peptides by professional APCs to T cells is responsible for bridging the gap between innate and acquired immunity (2, 3). The recruitment of Ag-specific IFN-{gamma}-secreting CD4+ T cells to the site of infection is a critical component of immunity to tuberculosis, as it leads to the release of soluble factors such as IFN-{gamma}, activating infected macrophages and inhibiting the replication of M. tuberculosis (4). Dendritic cells (DCs)3 are recognized as the most potent stimulators of naive T cells, as they are specialized for Ag uptake in the periphery, upon which they migrate via afferent lymphatics to the draining lymph nodes (DLNs) (5). During this process, DCs up-regulated chemokine receptors and costimulatory molecules that are required for migration and effective stimulation of T cells (1, 6). Mature DCs are defined by a high surface expression of MHC class II in conjunction with the expression of the integrin-{alpha}x chain, CD11c, and high surface expression of costimulatory molecules (7). Jiao et al. (8) showed that following high dose i.v. infection of mice with bacillus Calmette-Guérin (BCG), DCs are the most potent subset of cells at presenting Ag to splenic T cells. DCs rapidly distribute into the T cell zone after BCG infection, produce IL-12p40, and up-regulate costimulatory molecules including CD40, CD80, and CD86 (8). Depletion of CD11c+ cells in vivo delays the onset of an Ag-specific T cell response to M. tuberculosis and impairs the control of the infection (9). Similarly, depletion of activated macrophages resulted in impaired resistance of mice to pulmonary M. tuberculosis infection (10). Therefore, modulating APC function may prove an effective vaccine strategy to control infection with M. tuberculosis.

The cytokine GM-CSF plays a pleiotropic role in the generation, proliferation, and differentiation of hemopoietic progenitor cells toward cells of myeloid lineage such as monocytes/macrophages, DCs, neutrophils, and eosinophils (11). GM-CSF also acts locally as a proinflammatory cytokine, both in recruitment of leukocytes and the enhancement of APC function (12). GM-CSF can induce up-regulation of MHC class II and costimulatory molecules such as CD80 and CD86 on APC as well as increasing their phagocytic activity and stimulatory capacity (13, 14, 15, 16, 17). Coimmunization with or over-expression of GM-CSF enhances Ag-specific IFN-{gamma}-secreting T cells and increases the protection against a variety of infectious agents (18, 19, 20, 21). GM-CSF-deficient APCs are poor inducers of T cell responses, suggesting that the cytokine is required for optimal generation of T cell immunity (22, 23).

Recombinant BCG strains can be engineered to express immuno-stimulatory molecules such as cytokines and chemokines in a functional form (24, 25, 26, 27). This study aimed to modulate APC function by delivery of GM-CSF via the BCG vaccine. BCG-derived GM-CSF induced in vitro differentiation of APCs with a potent capacity to stimulate naive T cells. In vivo administration of the cytokine-producing strain led to an increase in APC numbers in lymph nodes that drain the site of immunization with elevated immunostimulatory function. BCG-expressing GM-CSF induced an early increase in IFN-{gamma} production, which was maintained up to 17 wk postimmunization. The heightened immune response after BCG-secreting murine GM-CSF (BCG:GM-CSF) immunization translated to increased protection against disseminated infection when mice were challenged with aerosol M. tuberculosis.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 Disclosures
 References
 
Bacterial strains and growth conditions

M. tuberculosis H37Rv (ATCC 27294) was grown in Proskauer and Beck liquid medium for 14 days at 37°C. Recombinant BCG strains were grown in Middlebrook 7H9 broth with 10% albumin-dextrose-catalase (ADC) enrichment (Difco Laboratories). When required, the antibiotics kanamycin (25 µg/ml) and/or hygromycin B (50 µg/ml) were added to liquid and/or solid medium for rBCG cultures. Mycobacteria were enumerated on Middlebrook 7H11 agar supplemented with 10% oleic acid-ADC enrichment (Difco Laboratories).

BCG:GM-CSF was a gift of Professor Rick Young, Whitehead Institute (Cambridge, MA) (25). Construction of the control BCG strain (BCG:Ct), harboring the pMV261 plasmid, has been previously described (27). To construct BCG expressing the OVA protein, residues 230–259 of OVA protein were amplified by PCR, the PCR product digested with BamHI and EcoRI, and ligated into the pJEX66 vector (28), resulting in pJEX84. A 1.1 kb fragment containing the Mycobacterium bovis HSP60 promoter, the M. tuberculosis Ag85B signal sequence, OVA250–345, and c-myc epitope tag was excised from pJEX84 and inserted into the mycobacterial integrative vector pNIP-40b (provided by Dr. Nathalie Winter, Pasteur Institute, Paris, France) to yield pJEX87. Plasmid JEX87 was transformed into BCG:Ct and BCG:GM-CSF as previously described (29).

Animals

Female C57BL/6 and male B6.SJL/PtprCa mice aged 6–8 wk were purchased from the Animal Research Centre. OT-II transgenic mice were bred in the Centenary Institute Animal Facility and were a gift from Professor Bill Heath (Walter and Eliza Hall Institute, Melbourne, Australia). Mice were maintained in specific pathogen-free conditions in the Centenary Institute Animal Facility under ethical approval from the Sydney University Animal Ethics Committee.

APC preparation and Ag presentation

Bone marrow progenitor cells were generated as previously described (30). For in vitro studies, rBCG strains were grown in 7H9 broth supplemented with 10% ADC (without albumin) for 14 days. Supernatants were collected, concentrated by freeze-drying, and protein resuspended in culture medium (RPMI 1640 supplemented with 100 µg/ml penicillin, 100 µg/ml streptomycin, 10 mM sodium bicarbonate, and 20 mM HEPES). Total protein concentration was determined by BCA protein concentration kit (Pierce) and BCG-derived GM-CSF was detected by ELISA (PeproTech). Approximately 1 ng/ml rGM-CSF derived from BCG was added to bone marrow progenitor cells, cells were cultured for 5 days at 37°C in 5% CO2, and then analyzed by flow cytometry. GM-CSF-derived APCs were infected with BCG at multiplicity of infection of 1 for 24 h and IL-12 release was measured by ELISA from culture supernatants. For Ag presentation studies, 1 x 105 of M. tuberculosis Ag85B p25240–258-specific CD4+ T cell hybridoma cells (T. M. Wozniak and W. J. Britton, manuscript in preparation) were cultured with BCG-infected APC at a ratio of 1:1 and IL-2 release was measured from culture supernatants after 72 h.

Immunization and M. tuberculosis challenge

For immunogenicity studies, 6- to 8-wk-old female C57BL/6 mice (n = 4) were infected via s.c. injection with 1 x 106 CFU of BCG:GM-CSF or BCG:Ct. At selected time-points, mice were sacrificed and spleens, DLNs (popliteal, inguinal, and para-aortic), and non-DLNs (NDLNs; cervical) from each group harvested for ex vivo restimulation and flow cytometry. Single cell suspensions of organs were prepared, incubated with anti-Fc{gamma}RIII/II (clone 2.4G2), and stained with Abs for the following markers using appropriate fluorochromes and concentrations (BD Pharmingen): CD11c, B220, MHC class II, CD40, CD80, and CD86. Cell viability was assessed by use of 4',6-diamidino-2-phenylindole dihydrochloride. Samples were processed using an LSR-II flow cytometer (BD Biosciences) and data analyzed using FlowJo analysis software (Tree Star).

For protection studies, 6- to 8-wk-old female C57BL/6 mice (n = 5) were infected via s.c. injection at the base of the tail (5 x 105 CFU) with BCG:GM-CSF or BCG:Ct. Twelve weeks postvaccination, mice were exposed to M. tuberculosis H37Rv (ATCC 27294) using a Middlebrook airborne infection apparatus (Glas-Col) with an infective dose of ~100 viable bacilli per lung. Four weeks following M. tuberculosis challenge, homogenized lungs and spleens were plated on supplemented Middlebrook 7H11 Bacto agar, and colonies were counted and expressed as log10 CFU.

Cytokine assays

Ag-specific IFN-{gamma}-secreting cells were measured by ELISPOT as previously described (29). For cytokine ELISA, plates were coated with anti-IL-12p40, anti-IFN-{gamma} (AN18), or anti-GM-CSF Abs (R&D Systems), supernatant samples and standards added, and anti-IL-12p40biotin (R&D Systems), anti-IFN-{gamma} biotin (XMG1.2), or anti-GM-CSF (R&D Systems) were added followed by streptavidin-HRP. Substrate solution was added and absorbance measured at a dual wavelength of 405/492 nm.

CFSE labeling and adoptive transfer

Lymph node cells from donor OT-II mice (Ly5.2+) were labeled with 5 µM CFSE as previously described (31). Approximately 5 x 105 cells were injected i.v. into B6.SJL/PtprCa (Ly5.1+) host mice and 1 day post-transfer, mice were infected s.c. with 1 x 106 CFU BCG:Ct-OVA or BCG:GM-CSF-OVA. On day 3 or 7 postvaccination, the activation of transgenic T cells in the DLNs, spleen, and NDLNs were analyzed by flow cytometry. Cells were stained with Abs for the following markers using appropriate fluorochromes and concentrations (BD Pharmingen): CD4, CD44, CD62L, CD45.1, and CD45.2. Before acquisition of the samples, 4',6-diamidino-2-phenylindole dihydrochloride was added to exclude dead cells. The CFSE profile of dividing cells was analyzed as described (32, 33).

Statistical analysis

Statistical analysis of the results from immunological and log-transformed bacterial counts were conducted using ANOVA. Fisher’s protected least significant difference ANOVA post hoc test was used for pair-wise comparison of multi-grouped data sets. Differences with p < 0.05 were considered significant.


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 Disclosures
 References
 
Biologically active GM-CSF secreted by BCG differentiates APC in vitro

The central role of APCs in the control of mycobacterial infection (8, 9), coupled with the well-established ability of GM-CSF to influence the differentiation and activation of APC subsets, suggested GM-CSF may serve to improve the efficacy of anti-tuberculosis vaccines. To explore this hypothesis, we made use of BCG:GM-CSF (25). We first determined whether GM-CSF secreted by BCG was able to influence the differentiation of APCs in established in vitro systems. Secretion of GM-CSF was only detected in the culture supernatant of BCG:GM-CSF and not with BCG:Ct (Fig. 1A). Culture of bone marrow progenitor cells with supernatants from BCG:GM-CSF for 5 days resulted in the generation of CD11c+/CD11b+ cells, which were observed at low numbers in cultures supplemented with BCG:Ct-derived supernatants or medium alone (Fig. 1B). APCs differentiated with BCG-derived GM-CSF were potent producers of IL-12p40 after BCG infection, whereas no IL-12 was detected in infected cells generated by BCG:Ct supernatants (Fig. 1C). In addition, progenitor cells infected with BCG were capable of presenting Ag to an M. tuberculosis Ag85B p25240–258-specific CD4+ T cell hybridoma (T. M. Wozniak and W. J. Britton, manuscript in preparation) as measured by the release of IL-2 from stimulated T cells (Fig. 1D). No IL-2 was detected in the supernatants from APC cultures generated with control BCG supernatants. Therefore GM-CSF derived from BCG was capable of generating APC in vitro with the capacity to stimulate the activation of naive CD4+ T cells.


Figure 1
View larger version (16K):
[in this window]
[in a new window]

  		FIGURE 1. Biologically active GM-CSF produced by BCG leads to differentiation of bone marrow progenitors. The production of GM-CSF from supernatants of 2-wk broth cultures was determined by ELISA (A). Concentrated supernatants from BCG:GM-CSF or BCG:Ct cultures were added to bone marrow progenitor cells and the generation of CD11c+/CDllb+ cells examined (B). The release of IL-12 by CD11c+/CD11b+ cells generated by BCG-derived GM-CSF was determined by ELISA (C). The ability of the same cells to present the p25 peptide from the M. tuberculosis Ag85B protein to a p25-specific CD4+ T cell hybridoma was assessed by the release of IL-2 after 72 h of coculture (D). The significances of differences between the groups were analyzed by ANOVA (*, p < 0.05). Data are the mean ± SEM for triplicate wells and are representative of two independent experiments. ND; Not detected.

 
Expression of GM-CSF by BCG increases APC numbers in the DLNs following immunization

As GM-CSF secreted by BCG was able to promote the in vitro differentiation of APCs, we next determined whether BCG:GM-CSF influenced the recruitment and/or differentiation of APCs following vaccination. After s.c immunization of mice with rBCG strains, the number of CD11c+MHCII+ cells in the DLN increased over the course of BCG infection when compared with naive control mice (Fig. 2A). Further, the influence of GM-CSF secreted by BCG led to an increase in CD11c+MHCII+ cells compared with mice vaccinated with the BCG:Ct, with the most significant increase occurring at day 7 postinfection. Examination of DC numbers in the spleens and NDLNs of mice vaccinated with either rBCG strain revealed no significant increase in DC numbers when compared with naive mice (Fig. 2, B and C).


Figure 2
View larger version (15K):
[in this window]
[in a new window]

  		FIGURE 2. Immunization with GM-CSF-secreting BCG leads to increased APC numbers in the DLNs. C57BL/6 mice were immunized by s.c. injection with 1 x 106 CFU of rBCG strains, and at 7, 14, and 56 days postimmunization, mice were sacrificed and the number of CD11c+MHCII+(A–C) or CD11c–CD11b+F480+ (D–F) cells in the DLNs (A and D), spleen (B and E), or NDLNs (C and F) determined by flow cytometry. Data are the mean ± SEM and are representative of one of two independent experiments. (•), BCG:GM-CSF; ({square}), BCG:Ct; ({diamond}), unvaccinated. The significances of differences were determined by ANOVA between BCG:Ct and BCG:GM-CSF (*, p < 0.05) and between BCG:GM-CSF and naive controls ({dagger}, p < 0.01).

 
We also observed changes in the number of cells with a macrophage phenotype (CD11c–CD11b+F480+) after vaccination with BCG:GM-CSF. At both 7 and 14 days postvaccination, significantly greater CD11c–CD11b+F480+ cells were present in the DLN compared with mice immunized with the BCG control strain (Fig. 2D). This increase was less apparent in the spleen and NDLN (Fig.s 2, E and F). Changes in cell number after BCG:GM-CSF were not observed in other cells of the innate immune response, as we observed no significant difference in the number of NK1.1+ cells after vaccination with BCG:GM-CSF or BCG:Ct (data not shown). These data indicate that that GM-CSF secreted by BCG increases APC numbers at sites draining the site of immunization but not at distant sites.

Influence of GM-CSF secretion by BCG on the activation state of DCs

To determine whether secretion of GM-CSF by BCG could influence the early activation of DCs in the DLN, expression of costimulatory molecules was analyzed on DCs. The two major DC subsets detected in murine lymph nodes were CD11cintMHCIIhigh and CD11chighMHCIIint populations (Fig. 3, A and B) (34). The CD11cintMHCIIhigh DC subset is tissue-derived with DCs migrating into the lymph node from the dermis and epidermis (35). Changes in the expression of the costimulatory molecules CD80 and CD86 on these DC populations were analyzed at day 3 postvaccination. Migratory DCs had a greater expression of both CD80 (Fig. 3C) and CD86 (Fig. 3E) in the DLN upon immunization with BCG compared with naive controls, which was further increased in mice immunized with BCG:GM-CSF. No significant increase in CD80 and CD86 was observed in this subset in NDLN from mice immunized with either rBCG strain (Fig. 3, D and F). Therefore GM-CSF altered the local activation of DCs after vaccination with rBCG.


Figure 3
View larger version (48K):
[in this window]
[in a new window]

  		FIGURE 3. Secretion of GM-CSF by BCG increases the activation states of DCs migrating into the DLNs. C57BL/6 mice were infected s.c. with 1 x 106 CFU of BCG:Ct or BCG:GM-CSF, and at day 3, the DLNs and NDLNs were harvested and DC populations analyzed by flow cytometry. DCs were identified based on the expression of CD11c and MHCII (A and B), and expression of CD80 (C and D) or CD86 (E and F) was examined on CD11cintMHCIIhigh cells in the DLN (C and E) or NDLN (D and F). unv, unvaccinated mice ({square}); Ct, BCG:Ct-vaccinated mice (Figure 3); GM-CSF, BCG-GM-CSF-vaccinated mice ({blacksquare}). The differences between the groups were determined by ANOVA (*, p < 0.05, **, p < 0.01; NS, Not significant) and data are representative of two independent experiments.

 
Expression of GM-CSF by BCG leads to enhanced activation of Ag-specific T cells in vivo

The increased capacity of BCG:GM-CSF to influence DC numbers and activation state may influence the resulting T cell response after vaccination. To examine this, we developed rBCG strains that expressed both GM-CSF and a truncated portion of the OVA protein. The expression of the OVA gene by rBCG was confirmed by Western blotting of BCG:GM-CSF-OVA and BCG:Ct-OVA cell lysates and revealed similar expression levels of the truncated OVA protein (Fig. 4A). This allowed direct in vivo comparison of the immunogenicity of the strains.


Figure 4
View larger version (21K):
[in this window]
[in a new window]

  		FIGURE 4. Influence of BCG-derived GM-CSF on the early activation of CD4+ T cells in the DLNs. A, Cell lysates of BCG:Ct encoding residues 230–259 of the OVA protein (lane 1, BCG:OVA) or BCG:GM-CSF expressing the same OVA fragment (lane 2, BCG:GM-OVA) were subject to immunoblotting with the anti-c-myc mAb 9E10. B, CFSE-labeled OT-II lymph node cells were adoptively transferred into B6.SJL/PtprCa host mice on day –1. On day 0, mice were left unvaccinated (unvac), or immunized s.c with BCG:Ct-OVA or BCG:GM-CSF-OVA, and the CFSE division profile of transferred OT-II cells (CD45.1–CD45.2+CD4+) in the DLN determined at day 7. The percentage of transferred OT-II cells recruited into division at day 3 (C) or 7 (D) post-rBCG vaccination was also determined. The proportion of CFSElow OT-II cells from the day 7 time-point displaying a CD62Llow phenotype is shown in E. C57BL/6 mice were immunized s.c. with 5 x 105 CFU of rBCG, and at day 7 postvaccination, the number of DLN cells secreting IFN-{gamma} in response to M. tuberculosis Ag85B peptide-25 was determined by ELISPOT (F). Differences between groups were determined by ANOVA (*, p < 0.05) and data are representative of two independent experiments.

 
CFSE-labeled CD4+ OT-II transgenic cells were adoptively transferred into B6.SJL/PtprCa recipients, mice were infected s.c. with rBCG strains expressing the truncated OVA gene, and the proliferation of Ag-specific T cells was analyzed. Proliferation of transferred OT-II cells in mice vaccinated with BCG:Ct-OVA and BCG:GM-CSF-OVA was observed at days 3 (not shown) and 7 postvaccination (Fig. 4B). All divided cells were CD44high, indicating that the cells had been activated (not shown). T cell priming was equivalent after vaccination with both rBCG strains, as no difference was observed in the proportion of OT-II cells recruited into cell division at day 3 (Fig. 4C) or 7 (Fig. 4D) postvaccination. However, vaccination with BCG:GM-CSF-OVA resulted in a significantly greater proportion of divided cells with a CD62Llow phenotype, suggesting the enhanced generation of effector memory T cells (Fig. 4E). This difference correlated with an increase in the frequency of Ag-specific IFN-{gamma}-secreting CD4+ T cells in the DLN of mice immunized with BCG:GM-CSF compared with those vaccinated with control BCG at 7 days postimmunization (Fig. 4F).

To determine whether early differences in T cell activation correlated with changes in long-term immune responses induced by BCG:GM-CSF, mice were immunized with BCG:GM-CSF or BCG:Ct and at 5 and 17 wk splenocytes were restimulated with BCG Ags ex vivo. At 5 wk postimmunization, splenocytes from mice vaccinated with BCG:GM-CSF had a nearly 3-fold increase in the number of IFN-{gamma}-secreting cells compared with the BCG:Ct (Fig. 5A). At 17 wk, the number of Ag-specific T cells had contracted when compared with the response at 5 wk in both rBCG vaccinated groups. However, the BCG:GM-CSF strain had stimulated a 4-fold greater number of BCG-specific T cells than control BCG (Fig. 5B). These results indicate that BCG-derived GM-CSF has a long-term effect of the activation and maintenance of Ag-specific T cells generated by vaccination.


Figure 5
View larger version (17K):
[in this window]
[in a new window]

  		FIGURE 5. Immunization with BCG:GM-CSF increases the protective effect BCG against dissemination of aerosol M. tuberculosis infection. C57BL/6 mice were immunized s.c. with 5 x 105 CFU of rBCG strains, and at 5 (A) and 17 (B) wk postimmunization, the release of IFN-{gamma} by splenocytes in response to BCG lysate was determined by ELISPOT. Data are expressed as mean ± SEM (*, p < 0.05) and are representative of two independent experiments. To assess protective efficacy, mice were left unvaccinated (unvac) or immunized s.c. with 5 x 105 CFU rBCG strains, and 12 wk postvaccination mice were challenged by aerosol M. tuberculosis H37Rv. Four weeks postchallenge, the bacterial loads in the lungs (C) and spleen (D) were determined. Data are the mean ± SEM for five mice and are representative of three independent experiments. The significances of differences between the groups were determined by ANOVA (*, p < 0.05, **, p < 0.005; NS, Not significant).

 
Expression of GM-CSF by BCG increases the protective effect of the vaccine against disseminated M. tuberculosis infection

To determine whether the influence of BCG-derived GM-CSF on APC number, DC activation, and T cell activation translated into increased protective efficacy, C57BL/6 mice were immunized with rBCG strains and aerosol challenged with M. tuberculosis. BCG:GM-CSF afforded ~1-log reduction in M. tuberculosis load in the lungs compared with unvaccinated mice (p < 0.01) (Fig. 5C). The level of protection afforded by BCG:GM-CSF was consistently greater than that following control BCG, however this difference did not reach statistically significance (Fig. 5C). Control BCG reduced the bacterial load in the spleen by ~1.5 log compared with unvaccinated mice (p < 0.01) (Fig. 5D). BCG:GM-CSF further increased protection in the spleen by ~1 log compared with control BCG, and this difference was statistically significant (p < 0.05) (Fig. 5D). Therefore the secretion of GM-CSF improved the capacity of BCG to control M. tuberculosis growth at a site of bacterial dissemination but not the primary site of infection.


   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
Disclosures
 References
 
A variety of approaches have been used to influence immune responses to vaccines (36). In this study, a rBCG strain secreting murine GM-CSF was used to determine whether modulating APC function would enhance the vaccine efficacy of BCG. GM-CSF was secreted by BCG in a functional form with the capacity to stimulate the generation of APCs from bone marrow progenitors in vitro. These APCs were capable of secreting IL-12p40 and activating naive CD4+ T cells (Fig. 1). This effect was also observed in vivo with BCG:GM-CSF stimulating early increases in the number of macrophages and DCs in lymph nodes draining the site of vaccination (Fig. 2). Interestingly, a similar increase in macrophage-like cells (F4/80+CD11b+) in the peritoneal exudate was observed by Burger et al. (14) following i.p. injection of an adenovirus encoding murine GM-CSF. Secretion of GM-CSF by BCG did not have a selective effect on any particular DC subpopulation defined by the expression of CD8, CD11b, and B220 (not shown). This parallels observations made by Vremec et al. (37) whereby over-expression of GM-CSF in transgenic mice increased the total number of DCs; however, the cytokine had no effect on the pro-portions of DC subtypes. Other studies, however, have demonstrated a preferential expansion of CD11bhighCD8a+ DC subsets following adenoviral production of GM-CSF or delivery of polyethylene glycol-modified GM-CSF (15, 17). The reasons for these differences are unclear but may reflect differences in the level of cytokine produced by vaccine vectors, and the mode of delivery and localization of GM-CSF-secreting constructs. Interestingly, the concentration of GM-CSF detected in culture supernatants of rBCG:GM-CSF is comparable to GM-CSF levels produced by murine or human cells infected with BCG (26, 38), and human PBMCs infected with BCG produce GM-CSF at relatively late time-points postinfection (38). Therefore the capacity of BCG:GM-CSF to deliver the cytokine early after vaccination may account for its potent effect on immunogenicity. We also observed increased activation of DCs in the DLN after BCG:GM-CSF activation compared with DCs obtained from unvaccinated or BCG:Ct vaccinated mice (Fig. 3). This further supports the potential of BCG-derived murine mGM-CSF to influence anti-mycobacterial immunity and is consistent with previous studies which demonstrated that delivery of GM-CSF to mice or humans by viral vectors or tumor cells increased the activation of DCs (17, 39, 40).

BCG:GM-CSF was able to significantly increase anti-mycobacterial T cell responses even though the recombinant strain secreted relatively low levels of murine GM-CSF (Fig. 1). In addition, we observed no gross pathological changes in mice vaccinated with BCG:GM-CSF compared with those vaccinated with control BCG (data not shown). By contrast, transgenic mice over-expressing GM-CSF demonstrated enlarged and histologically abnormal livers and spleens as well as increased number of macrophages and granulocytes in the peripheral blood (41). Delivery of polyethylene glycol-conjugated GM-CSF to mice impaired resistance against M. tuberculosis infection (42), and transgenic mice over-expressing GM-CSF were unable to control infection with M. tuberculosis at extended time-points postchallenge because of an inability of these mice to develop a normal granulomatous response (43). In humans, however, the use of tumor cells engineered to over-express GM-CSF have extensively been tested in clinical trials, and for the most part the regime is well tolerated (44). Further, recombinant human GM-CSF has been tested in combination with rifampin/isoniazid to treat patients with active tuberculosis with no adverse effects (45). Thus the capacity to deliver transiently low levels of immunostimulatory cytokines via BCG may be advantageous in invoking protective immune responses without negatively influencing pathology.

Secretion of GM-CSF by BCG had a potent effect on T cell immunity with early expansion of T cells after vaccination (Fig. 4), enhanced and sustained priming of anti-mycobacterial IFN-{gamma}-secreting T cells (Fig. 5), and improved protection against disseminated M. tuberculosis after aerosol challenge (Fig. 5). Therefore the effects of rBCG-derived GM-CSF on APC function translated into the generation of protective T cell immunity. The increase in early activation of T cell responses afforded by BCG:GM-CSF may also relate to the capacity of the cytokine to stimulating neutrophil phagocytosis, and neutrophils have been implicated in the transfer of BCG from the site of vaccination to the DLNs for presentation by professional APCs (46, 47). Other cytokines are secreted by rBCG in a functional form and also increase anti-mycobacterial T cell immunity after vaccination of mice (24, 25, 26). We have compared the protective effect of BCG-secreting IL-2 and IL-18 against M. tuberculosis infection and found that these rBCG strains did not afford better protection than the BCG:Ct (data not shown). This suggests that delivery of cytokines such as GM-CSF, which exert their influence on T cell immunity via their effect on APC function, is more protective in this model of murine M. tuberculosis infection. These findings provide important information for the rational design of effective vaccines against tuberculosis and potentially other chronic intracellular infections.

The improved capacity of BCG:GM-CSF to stimulate APC activation and T cell immunity within the DLNs was associated with improved protection against dissemination of aerosol M. tuberculosis infection (Fig. 5). This is supported by the increased IFN-{gamma} responses in the spleen after BCG:GM-CSF delivery, which was apparent to at least 17 wk postvaccination. However, it appears pulmonary immunity was not sufficiently improved by systemic delivery of BCG:GM-CSF, as both BCG:Ct and BCG:GM-CSF afforded equivalent levels of protection in the lung (Fig. 5). Interestingly, systemic delivery of adenovirus encoding GM-CSF was an effective adjuvant to increase anti-chlamydial immunity in the spleen; however, this did not result in increased protection against pulmonary challenge with Chlamydia trachomatis (48). By contrast, intrapulmonary delivery of adenovirus:GM-CSF afforded significant protection against C. trachomatis in the lung (48). Continuing studies are aimed at assessing the effect of pulmonary delivery of BCG:GM-CSF on protection against M. tuberculosis infection.

In summary, this report demonstrates that secretion of GM-CSF by rBCG modifies immunity in mice and improves protection against disseminated M. tuberculosis infection. Murine and human GM-CSF display limited biological cross-reactivity (49), which suggests modification of the recombinant strain would be required for use in humans. However the potent activity of human GM-CSF on APC function (44), coupled with the favorable safety profile of studies employing human GM-CSF in clinical trials (44, 45), suggests further study of this vaccine as a candidate to protect against tuberculosis in human populations is warranted.


   Acknowledgments
 
We thank Prof. Rick Young (Whitehead Institute, Cambridge, MA) for providing the BCG:GM-CSF strain, Dr. Nathalie Winter (Institut Pasteur, Paris, France) for the pNIP-40 vector, and Prof. Bill Heath (Walter and Eliza Hall Institute, Melbourne, Australia) for the OT-II transgenic mice.


   Disclosures
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Disclosures
References
 
The authors have no financial conflict of interest.


   Footnotes
 
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1 This work was supported by the National Health and Medical Research Council of Australia. J.A.T was supported by a National Health and Medical Research Council Career Development Award, A.A.R was supported by an Australian Postgraduate Award, and T.M.W. was the recipient of the University of Sydney Faculty of Medicine Postgraduate Scholarship. Back

2 Address correspondence and reprint requests to Dr. James A. Triccas, Discipline of Infectious Diseases and Immunology, University of Sydney, Camperdown, NSW 2006, Australia. E-mail address: jamiet{at}infdis.usyd.edu.au Back

3 Abbreviations used in this paper: DC, dendritic cell; DLN, draining lymph nodes; NDLN, nonDLN; BCG, bacillus Calmette-Guérin; ADC, albumin-dextrose-catalase; BCG:GM-CSF, BCG-secreting murine GM-CSF; BCG:Ct, control BCG strain. Back

Received for publication August 27, 2007. Accepted for publication October 15, 2007.


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Disclosures
 References
 

  1. Bhatt, K., S. P. Hickman, P. Salgame. 2004. Cutting edge: a new approach to modeling early lung immunity in murine tuberculosis. J. Immunol. 172: 2748-2751. [Abstract/Free Full Text]
  2. Flynn, J. L., J. Chan. 2001. Immunology of tuberculosis. Ann. Rev. Immunol. 19: 93-129. [Medline]
  3. Hickman, S. P., J. Chan, P. Salgame. 2002. Mycobacterium tuberculosis induces differential cytokine production from dendritic cells and macrophages with divergent effects on naive T cell polarization. J. Immunol. 168: 4636-4642. [Abstract/Free Full Text]
  4. Cooper, A. M., D. K. Dalton, T. A. Stewart, J. P. Griffin, D. G. Russell, I. M. Orme. 1993. Disseminated tuberculosis in interferon {gamma} gene-disrupted mice. J. Exp. Med. 178: 2243-2247. [Abstract/Free Full Text]
  5. Hart, D. N.. 1997. Dendritic cells: unique leukocyte populations which control the primary immune response. Blood 90: 3245-3287. [Free Full Text]
  6. Banchereau, J., R. M. Steinman. 1998. Dendritic cells and the control of immunity. Nature 392: 245-252. [Medline]
  7. Shortman, K., Y. J. Liu. 2002. Mouse and human dendritic cell subtypes. Nat. Rev. Immunol. 2: 151-161. [Medline]
  8. Jiao, X., R. Lo-Man, P. Guermonprez, L. Fiette, E. Deriaud, S. Burgaud, B. Gicquel, N. Winter, C. Leclerc. 2002. Dendritic cells are host cells for mycobacteria in vivo that trigger innate and acquired immunity. J. Immunol. 168: 1294-1301. [Abstract/Free Full Text]
  9. Tian, T., J. Woodsworth, M. Skold, S. M. Behar. 2005. In vivo depletion of CD11c+ cells delays the CD4+ T cell response to Mycobacterium tuberculosis and exacerbates the outcome of infection. J. Immunol. 175: 3268-3272. [Abstract/Free Full Text]
  10. Leemans, J. C., T. Thepen, S. Weijer, S. Florquin, N. van Rooijen, J. G. van de Winkel, T. van der Poll. 2005. Macrophages play a dual role during pulmonary tuberculosis in mice. J. Infect. Dis. 191: 65-74. [Medline]
  11. Gasson, J. C.. 1991. Molecular physiology of granulocyte-macrophage colony-stimulating factor. Blood 77: 1131-1145. [Free Full Text]
  12. Shi, Y., C. H. Liu, A. I. Roberts, J. Das, G. Xu, G. Ren, Y. Zhang, L. Zhang, Z. R. Yuan, H. S. Tan, et al 2006. Granulocyte-macrophage colony-stimulating factor (GM-CSF) and T-cell responses: what we do and don’t know. Cell Res. 16: 126-133. [Medline]
  13. Larsen, C. P., S. C. Ritchie, R. Hendrix, P. S. Linsley, K. S. Hathcock, R. J. Hodes, R. P. Lowry, T. C. Pearson. 1994. Regulation of immunostimulatory function and costimulatory molecule (B7-1 and B7-2) expression on murine dendritic cells. J. Immunol. 152: 5208-5219. [Abstract]
  14. Burger, J. A., S. M. Baird, H. C. Powell, S. Sharma, D. J. Eling, T. J. Kipps. 2000. Local and systemic effects after adenoviral transfer of the murine granulocyte-macrophage colony-stimulating factor gene into mice. Br. J. Haematol. 108: 641-652. [Medline]
  15. Daro, E., B. Pulendran, K. Brasel, M. Teepe, D. Pettit, D. H. Lynch, D. Vremec, L. Robb, K. Shortman, H. J. McKenna, et al 2000. Polyethylene glycol-modified GM-CSF expands CD11bhighCD11chigh but not CD11blowCD11chigh murine dendritic cells in vivo: a comparative analysis with Flt3 ligand. J. Immunol. 165: 49-58. [Abstract/Free Full Text]
  16. Daro, E., E. Butz, J. Smith, M. Teepe, C. R. Maliszewski, H. J. McKenna. 2002. Comparison of the functional properties of murine dendritic cells generated in vivo with Flt3 ligand, GM-CSF and Flt3 ligand plus GM-SCF. Cytokine 17: 119-130. [Medline]
  17. Miller, G., V. G. Pillarisetty, A. B. Shah, S. Lahrs, Z. Xing, R. P. DeMatteo. 2002. Endogenous granulocyte-macrophage colony-stimulating factor overexpression in vivo results in the long-term recruitment of a distinct dendritic cell population with enhanced immunostimulatory function. J. Immunol. 169: 2875-2885. [Abstract/Free Full Text]
  18. Weiss, W. R., K. J. Ishii, R. C. Hedstrom, M. Sedegah, M. Ichino, K. Barnhart, D. M. Klinman, S. L. Hoffman. 1998. A plasmid encoding murine granulocyte-macrophage colony-stimulating factor increases protection conferred by a malaria DNA vaccine. J. Immunol. 161: 2325-2332. [Abstract/Free Full Text]
  19. Siddiqui, A. A., T. Phillips, H. Charest, R. B. Podesta, M. L. Quinlin, J. R. Pinkston, J. D. Lloyd, M. Paz, R. M. Villalovos, J. Pompa. 2003. Induction of protective immunity against Schistosoma mansoni via DNA priming and boosting with the large subunit of calpain (Sm-p80): adjuvant effects of granulocyte-macrophage colony-stimulating factor and interleukin-4. Infect. Immun. 71: 3844-3851. [Abstract/Free Full Text]
  20. Parker, J. N., L. A. Pfister, D. Quenelle, G. Y. Gillespie, J. M. Markert, E. R. Kern, R. J. Whitley. 2006. Genetically engineered herpes simplex viruses that express IL-12 or GM-CSF as vaccine candidates. Vaccine 24: 1644-1652. [Medline]
  21. Qiu, J. T., T. C. Chang, C. T. Lin, Y. M. Chen, F. Q. Li, Y. K. Soong, C. H. Lai. 2007. Novel codon-optimized GM-CSF gene as an adjuvant to enhance the immunity of a DNA vaccine against HIV-1 Gag. Vaccine 25: 253-263. [Medline]
  22. Park, L. S., D. Friend, S. Gillis, D. L. Urdal. 1986. Characterization of the cell surface receptor for granulocyte-macrophage colony-stimulating factor. J. Biol. Chem. 261: 4177-4183. [Abstract/Free Full Text]
  23. Wada, H., Y. Noguchi, M. W. Marino, A. R. Dunn, L. J. Old. 1997. T cell functions in granulocyte/macrophage colony-stimulating factor deficient mice. Proc. Natl. Acad. Sci. USA 94: 12557-12561. [Abstract/Free Full Text]
  24. O’Donnell, M. A., A. Aldovini, R. B. Duda, H. Yang, A. Szilvasi, R. A. Young, W. C. DeWolf. 1994. Recombinant Mycobacterium bovis BCG secreting functional interleukin-2 enhances {gamma} interferon production by splenocytes. Infect. Immun. 62: 2508-2514. [Abstract/Free Full Text]
  25. Murray, P. J., A. Aldovini, R. A. Young. 1996. Manipulation and potentiation of anti-mycobacterial immunity using recombinant bacille Calmette-Guerin strains that secrete cytokines. Proc. Natl. Acad. Sci. USA 93: 934-939. [Abstract/Free Full Text]
  26. Luo, Y., H. Yamada, X. Chen, A. A. Ryan, D. P. Evanoff, J. A. Triccas, M. A. O’Donnell. 2004. Recombinant Mycobacterium bovis bacillus Calmette-Guerin (BCG) expressing mouse IL-18 augments Th1 immunity and macrophage cytotoxicity. Clin. Exp. Immunol. 137: 24-34. [Medline]
  27. Ryan, A. A., J. M. Spratt, W. J. Britton, J. A. Triccas. 2007. Secretion of functional monocyte chemotactic protein-3 by recombinant Mycobacterium bovis BCG attenuates vaccine virulence and maintains protective efficacy against M. tuberculosis infection. Infect. Immun. 75: 523-526. [Abstract/Free Full Text]
  28. Spratt, J., A. A. Ryan, W. J. Britton, J. A. Triccas. 2005. Epitope-tagging vectors for the expression and detection of recombinant proteins in mycobacteria. Plasmid 53: 269-273. [Medline]
  29. Palendira, U., A. Kamath, C. Feng, E. Martin, P. Chaplin, J. Triccas, W. Britton. 2002. Co-expression of interleukin-12 chains by a self splicing vector increases the protective cellular immune response of DNA and BCG vaccines against Mycobacterium tuberculosis. Infect. Immun. 70: 1949-1956. [Abstract/Free Full Text]
  30. Wozniak, T. M., A. A. Ryan, J. A. Triccas, W. J. Britton. 2006. Plasmid interleukin-23 (IL-23), but not plasmid IL-27, enhances the protective efficacy of a DNA vaccine against Mycobacterium tuberculosis infection. Infect. Immun. 74: 557-565. [Abstract/Free Full Text]
  31. Fulcher, D. A., A. B. Lyons, S. L. Korn, M. C. Cook, C. Koleda, C. Parish, B. Fazekas de St. Groth, A. Basten. 1996. The fate of self-reactive B cells depends primarily on the degree of antigen receptor engagement and availability of T cell help. J. Exp. Med. 183: 2313-2328. [Abstract/Free Full Text]
  32. Fazekas de St. Groth, B., A. L. Smith, W. P. Koh, L. Girgis, M. C. Cook, P. Bertolino. 1999. Carboxyfluorescein diacetate succinimidyl ester and the virgin lymphocyte: a marriage made in heaven. Immunol. Cell Biol. 77: 530-538. [Medline]
  33. Shklovskaya, E., B. Fazekas de St. Groth. 2006. Severely impaired clonal deletion of CD4+ T cells in low-dose irradiated mice: role of T cell antigen receptor and IL-7 receptor signals. J. Immunol. 177: 8320-8330. [Abstract/Free Full Text]
  34. Salomon, B., J. L. Cohen, C. Masurier, D. Klatzmann. 1998. Three populations of mouse lymph node dendritic cells with different origins and dynamics. J. Immunol. 160: 708-717. [Abstract/Free Full Text]
  35. Villadangos, J. A., W. R. Heath. 2005. Life cycle, migration and antigen presenting functions of spleen and lymph node dendritic cells: limitations of the Langerhans cells paradigm. Semin. Immunol. 17: 262-272. [Medline]
  36. Chabalgoity, J. A., A. Baz, A. Rial, S. Grille. 2007. The relevance of cytokines for development of protective immunity and rational design of vaccines. Cytokine Growth Factor Rev. 18: 195-207. [Medline]
  37. Vremec, D., G. J. Lieschke, A. R. Dunn, L. Robb, D. Metcalf, K. Shortman. 1997. The influence of granulocyte/macrophage colony-stimulating factor on dendritic cell levels in mouse lymphoid organs. Eur. J. Immunol. 27: 40-44. [Medline]
  38. Luo, Y., X. Chen, M. A. O’Donnell. 2007. Mycobacterium bovis bacillus Calmette-Guerin (BCG) induces human CC- and CXC-chemokines in vitro and in vivo. Clin. Exp. Immunol. 147: 370-378. [Medline]
  39. Li, B., M. J. Van Roey, K. Jooss. 2007. Recombinant IL-7 enhances the potency of GM-CSF-secreting tumor cell immunotherapy. Clin. Immunol. 123: 155-165. [Medline]
  40. Dessureault, S., D. Noyes, D. Lee, M. Dunn, W. Janssen, A. Cantor, E. Sotomayor, J. Messina, S. J. Antonia. 2007. A phase-I trial using a universal GM-CSF-producing and CD40L-expressing bystander cell line (GM.CD40L) in the formulation of autologous tumor cell-based vaccines for cancer patients with stage IV disease. Ann. Surg. Oncol. 14: 869-884. [Medline]
  41. Burke, B., A. Pridmore, N. Harraghy, A. Collick, J. Brown, T. Mitchell. 2004. Transgenic mice showing inflammation-inducible overexpression of granulocyte macrophage colony-stimulating factor. Clin. Diagn. Lab. Immunol. 11: 588-598. [Medline]
  42. Alaniz, R. C., S. Sandall, E. K. Thomas, C. B. Wilson. 2004. Increased dendritic cell numbers impair protective immunity to intracellular bacteria despite augmenting antigen-specific CD8+ T lymphocyte responses. J. Immunol. 172: 3725-3735. [Abstract/Free Full Text]
  43. Gonzalez-Juarrero, M., J. M. Hattle, A. Izzo, A. P. Junqueira-Kipnis, T. S. Shim, B. C. Trapnell, A. M. Cooper, I. M. Orme. 2005. Disruption of granulocyte macrophage-colony stimulating factor production in the lungs severely affects the ability of mice to control Mycobacterium tuberculosis infection. J. Leukocyte Biol. 77: 914-922. [Abstract/Free Full Text]
  44. Chang, D. Z., W. Lomazow, C. Joy Somberg, R. Stan, M. A. Perales. 2004. Granulocyte-macrophage colony stimulating factor: an adjuvant for cancer vaccines. Hematology 9: 207-215. [Medline]
  45. Pedral-Sampaio, D. B., E. M. Netto, C. Brites, A. C. Bandeira, C. Guerra, M. G. Barberin, R. Badaro. 2003. Use of Rhu-GM-CSF in pulmonary tuberculosis patients: results of a randomized clinical trial. Braz. J. Infect. Dis. 7: 245-252. [Medline]
  46. Kletter, Y., I. Bleiberg, D. W. Golde, I. Fabian. 1989. Antibody to Mol abrogates the increase in neutrophil phagocytosis and degranulation induced by granulocyte-macrophage colony-stimulating factor. Eur. J. Haematol. 43: 389-396. [Medline]
  47. Abadie, V., E. Badell, P. Douillard, D. Ensergueix, P. J. Leenen, M. Tanguy, L. Fiette, S. Saeland, B. Gicquel, N. Winter. 2005. Neutrophils rapidly migrate via lymphatics after Mycobacterium bovis BCG intradermal vaccination and shuttle live bacilli to the draining lymph nodes. Blood 106: 1843-1850. [Abstract/Free Full Text]
  48. Lu, H., Z. Xing, R. C. Brunham. 2002. GM-CSF transgene-based adjuvant allows the establishment of protective mucosal immunity following vaccination with inactivated Chlamydia trachomatis. J. Immunol. 169: 6324-6331. [Abstract/Free Full Text]
  49. Miyajima, A., A. L. Mui, T. Ogorochi, K. Sakamaki. 1993. Receptors for granulocyte-macrophage colony-stimulating factor, interleukin-3, and interleukin-5. Blood 82: 1960-1974. [Free Full Text]



This article has been cited by other articles:


Home page
 J. Immunol.Home page
A. A. Ryan, J. K. Nambiar, T. M. Wozniak, B. Roediger, E. Shklovskaya, W. J. Britton, B. Fazekas de St. Groth, and J. A. Triccas
Antigen Load Governs the Differential Priming of CD8 T Cells in Response to the Bacille Calmette Guerin Vaccine or Mycobacterium tuberculosis Infection
J. Immunol., June 1, 2009; 182(11): 7172 - 7177.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow A correction has been published
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow Request Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ryan, A. A.
Right arrow Articles by Triccas, J. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ryan, A. A.
Right arrow Articles by Triccas, J. A.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS