Inclusion Criteria:

* All patients must have a histologically or cytologically proven diagnosis of adenocarcinoma of the prostate; note: If there is no formal biopsy report documenting the diagnosis of prostate cancer, the patient can be allowed on trial if the PSA level is at least 20, and there are at least three definitive metastatic lesions seen on scan; all patients must have had metastatic (M1) disease as evidenced by soft tissue and/or bony metastases prior to androgen deprivation therapy initiation; patients must have at least one of the following at the time they started androgen deprivation therapy:

* Visceral disease (liver, lung, or other viscera)
* Bone metastases to sites in either the axial (spine, pelvis, ribs, or skull) and/or the appendicular (clavicle, humerus, or femur) skeleton
* Lymph node disease not considered to be encompassed within a single radiotherapy port (e.g., above the aortic bifurcation, etc.)
* Patients who have measurable disease must have radiographic assessment (at least an abdominal/pelvic computed tomography \[CT\]) within 28 days prior to registration; non-measurable disease must also be assessed (e.g., bone scan) in all patients within 56 days prior to registration; all disease must be assessed and documented on the Baseline Tumor Assessment Form
* Patients must have a PSA \>= 5 ng/mL obtained within 90 days prior to initiation of androgen deprivation therapy
* Patients with known brain metastases are not eligible; brain imaging studies are not required for eligibility if the patient has no neurologic signs or symptoms, but if brain imaging studies are performed, they must be negative for disease
* Patient must have had no more than 30 days of prior medical castration for metastatic prostate cancer (prior androgen deprivation therapy is allowed if it was received with curative intent in the neoadjuvant, concurrent, and/or adjuvant fashion and at least 2 years have elapsed since completion of androgen deprivation therapy); the start date of medical castration is considered the day the patient first received an injection of a LHRH agonist, not an oral antiandrogen; if the method of castration is luteinizing hormone releasing hormone (LHRH) agonists (i.e., leuprolide or goserelin), the patient must be willing to continue the use of LHRH agonists and add bicalutamide for combined androgen deprivation therapy (ADT) during protocol treatment; the 30 day window begins from the date of receiving the LHRH agonist, not the oral antiandrogen; if the patient was on a different antiandrogen (e.g. flutamide), the patient must be willing to switch over to bicalutamide; patients must not have received bilateral orchiectomy; patients must not have received or be planning to receive LHRH antagonists (i.e., Degarelix); however, if the patient was initiated on a LHRH antagonist within the 30 day window and is willing to switch to a LHRH agonist with bicalutamide, he may enroll on the late induction group
* Patients who have not already started androgen deprivation therapy must be offered the opportunity to participate in the translational medicine studies; once a patient has started any form of antiandrogen (i.e., either bicalutamide or LHRH agonist), he is not eligible for any translational medicine studies
* Patients must not have received any prior cytotoxic chemotherapy for metastatic prostate cancer; prior cytotoxic chemotherapy with curative intent in the neoadjuvant or adjuvant setting is allowed; patients must not have received any prior treatment with agents that directly inhibit IGF or IGFRs
* Patients must not have received prior strontium-89, rhenium-186, rhenium-188, or samarium-153 radionuclide therapy within 28 days prior to registration
* Patients may have received prior radiation therapy or biologic therapy (e.g. vaccines, immunotherapy, anti-sense, small molecules, monoclonal antibodies); however, at least 28 days must have elapsed since completion of therapy and patient must have recovered from all side effects
* Patients may have received prior surgery; for all major surgeries, at least 28 days must have elapsed since completion and patient must have recovered from all side effects
* Leukocytes \>= 3,000 mcL
* Absolute neutrophil count (ANC) \>= 1,500 mcL
* Hemoglobin \>= 9 g/dL
* Platelets \>= 100,000/mcL
* Bilirubin =\< 1.5 times the institutional upper limit of normal (ULN) (unless documented Gilbert's disease)
* Serum glutamic oxaloacetic transaminase (SGOT) (aspartate aminotransferase \[AST\]) and serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase \[ALT\]) =\< 3 times the institutional ULN, or =\< 5 times the institutional ULN if liver metastases are present
* Creatinine =\< 2.0 x the institutional ULN or calculated creatinine clearance \>= 40 mL/min
* International normalized ratio (INR) =\< 1.5
* Partial thromboplastin time (PTT) no more than 5 seconds above the institutional ULN
* Patients receiving prophylactic low dose coumadin or low molecular weight heparin are eligible as long as they meet these coagulation criteria; patients requiring full-dose (therapeutic) anticoagulation are eligible provided that they have been on a stable dose of anticoagulation and the coagulation parameters are stable within the therapeutic range (e.g., INR 2-3 for patients on therapeutic warfarin)
* Patients must have a hemoglobin A1c (HgA1c) =\< 7% AND fasting glucose of \< 160 mg/dL or below the institutional ULN within 14 days prior to registration; patients with diabetes mellitus who meet this criterion must be on a stable dietary or therapeutic regimen for this condition
* Patients must not have a history of symptomatic congestive heart failure or a known ejection fraction (left ventricular ejection fraction \[LVEF\]) that is \>= 10% below the lower limit of normal (LLN); if left ventricular (LV) dysfunction is suspected, but not confirmed by review of past medical history, a multi gated acquisition scan (MUGA) or echocardiogram must be obtained within 90 days prior to registration
* Patient must not have a history of allergic reaction attributed to compounds of similar chemical or biologic composition to IMC-A12; patients must not have received prior chimerized or murine monoclonal antibody therapy
* Patients must have a Zubrod performance status of 0 - 2; Zubrod performance status 3 will be allowed if from bone pain only
* Patients with human immunodeficiency virus (HIV) positivity requiring antiretroviral therapy are not eligible for this study
* Patients must have no plans to receive concurrent chemotherapy, hormonal therapy (other than the LHRH agonist and oral anti-androgen), radiotherapy, immunotherapy or any other type of therapy for treatment of cancer while on this protocol treatment; concurrent bone targeting agents that do not have effect on PSA (i.e. denosumab or zoledronic acid) are allowed
* Patients must have no plans to receive concurrent five-alpha reductase inhibitors (e.g. finasteride and dutasteride), ketoconazole, diethylstilbestrol/DES, or other estrogen-based therapy while on this protocol treatment
* No other prior malignancy is allowed except for the following: adequately treated basal cell or squamous cell skin cancer, adequately treated stage I or II cancer from which the patient is currently in complete remission, or any other cancer from which the patient has been disease-free for 5 years
* Men of reproductive potential must have agreed to use an effective contraceptive method while receiving treatment on this study and for at least three months after protocol treatment ends
* All patients must be informed of the investigational nature of this study and must sign and give written informed consent in accordance with institutional and federal guidelines
* As part of the OPEN registration process, the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system