Inclusion Criteria:

1. Pathologically (histologically) proven diagnosis of prostatic adenocarcinoma, at intermediate risk for recurrence, within 180 days prior to registration as determined by having one or more of the following intermediate-risk features: Gleason score 7; prostate-specific antigen (PSA) >10 but ≤20; clinical stage T2b-T2c.

* Patients previously diagnosed with low risk (Gleason score ≤ 6, clinical stage < T2a, and PSA < 10) prostate cancer undergoing active surveillance who are re-biopsied and found to have intermediate risk disease according to the protocol criteria are eligible for enrollment within 180 days of the repeat biopsy procedure.
2. Clinically negative lymph nodes as established by imaging (pelvic +/- abdominal CT or MRI), nodal sampling, or dissection within 60 days prior to registration, except as noted immediately below:

* Patients with a single intermediate risk factor only do not require abdominopelvic imaging, but these studies may be obtained at the discretion of the treating physician. Patients with 2 or 3 risk factors are required to undergo pelvic +/- abdominal CT or MRI.
* Patients with lymph nodes equivocal or questionable by imaging are eligible without biopsy if the nodes are ≤1.5 cm; any node larger than this on imaging will require negative biopsy for eligibility.
3. No evidence of bone metastases (M0) on bone scan within 60 days prior to registration.

* Bone scan is not required for patients enrolled with a single intermediate risk factor only, but this scan may be obtained at the discretion of the treating physician. Patients with 2 or 3 risk factors will require a negative bone scan for eligibility.
* Equivocal bone scan findings are allowed if plain film x-rays are negative for metastasis.
4. History/physical examination (to include, at a minimum, digital rectal examination of the prostate and examination of the skeletal system and abdomen, and formal comorbidity assessment via the ACE-27 instrument) within 60 days prior to registration.
5. Zubrod Performance Status 0-1
6. Age ≥ 18
7. Baseline serum PSA value performed with an FDA-approved assay (e.g., Abbott, Hybritech) within 60 days prior to registration

* Study entry PSA must not be obtained during the following time frames: (1) 10-day period following prostate biopsy; (2) following initiation of ADT; (3) within 30 days after discontinuation of finasteride; or (4) within 90 days after discontinuation of dutasteride.
8. For patients undergoing brachytherapy only: complete blood count (CBC)/differential obtained within 60 days prior to registration, with adequate bone marrow function defined as follows:

* Absolute neutrophil count (ANC) ≥ 1,800 cells/mm3
* Platelets ≥ 100,000 cells/mm3
* Hemoglobin (Hgb) ≥ 8.0 g/dl (Note: The use of transfusion or other intervention to achieve Hgb ≥ 8.0 g/dl is acceptable.)
9. Patient must be able to provide study-specific informed consent prior to study entry.

Exclusion Criteria:

1. Patients with Gleason Score ≥ 8; PSA > 20; OR Clinical Stage ≥ T3 are ineligible for this trial.

* Should findings of extracapsular extension or seminal vesicle invasion be noted on prostate MRI, this study, if used, will not render patients ineligible for accrual to this protocol. Primary tumor staging for eligibility purposes is to be based on palpable or core biopsy evidence only with respect to extracapsular extension or seminal vesicle involvement.
2. Patients with all three intermediate risk factors who also have ≥ 50% of the number of their biopsy cores positive for cancer are ineligible for this trial.
3. Prior invasive malignancy (except non-melanomatous skin cancer) or hematological (e.g., leukemia, lymphoma, myeloma) malignancy unless disease free for a minimum of 5 years (prior diagnoses of carcinoma in situ are permitted)
4. Prior radical surgery (prostatectomy), high-intensity focused ultrasound (HIFU) or cryosurgery for prostate cancer
5. Prior hormonal therapy, such as LHRH agonists (e.g., goserelin, leuprolide), antiandrogens (e.g., flutamide, bicalutamide), estrogens (e.g., DES), or bilateral orchiectomy
6. Use of finasteride within 30 days prior to registration
7. Use of dutasteride within 90 days prior to registration
8. Prior or concurrent cytotoxic chemotherapy for prostate cancer; prior chemotherapy for a different cancer is permitted.
9. Prior RT, including brachytherapy, to the region of the study cancer that would result in overlap of RT fields

* Any patient undergoing brachytherapy must have transrectal ultrasound confirmation of prostate volume <60 cc, American Urological Association Symptom Index (AUA-SI) score ≤15 within 60 days of registration, and no history of prior transurethral resection of the prostate (TURP); prior TURP is permitted for patients who receive EBRT only)
10. Severe, active co-morbidity, defined as follows:

* Unstable angina and/or congestive heart failure requiring hospitalization within the last 6 months
* Transmural myocardial infarction within the last 6 months
* Acute bacterial or fungal infection requiring intravenous antibiotics at the time of registration
* Chronic Obstructive Pulmonary Disease exacerbation or other respiratory illness requiring hospitalization or precluding study therapy within 30 days before registration
* Hepatic insufficiency resulting in clinical jaundice and/or coagulation defects; note, however, that laboratory tests for liver function and coagulation parameters are not required for entry into this protocol.
* Acquired Immune Deficiency Syndrome (AIDS) based upon current Centers for Disease Control and Prevention (CDC) definition; note, however, that HIV testing is not required for entry into this protocol. While the treatment employed in this study is not significantly immunosuppressive, it is felt that a diagnosis of AIDS associated with prostate cancer is likely to impact this study's primary endpoint of overall survival. Patients who are HIV seropositive but do not meet criteria for diagnosis of AIDS are eligible for study participation.
11. Men who are sexually active with a woman of child-bearing potential and not willing/able to use medically acceptable forms of contraception (e.g., surgical, barrier, medicinal) during protocol treatment and during the first 3 months after cessation of protocol treatment; this exclusion is necessary because the treatment involved in this study may be significantly teratogenic.