Inclusion Criteria:

* Pathologically confirmed primary adenocarcinoma of the esophagus that involves the mid (up to 25 cm), distal, or esophagogastric junction; the cancer may involve the stomach up to 5 cm
* Endoscopy with biopsy
* PRIOR TO STEP 1 REGISTRATION BUT WITHIN 56 DAYS PRIOR TO STEP 2 REGISTRATION
* Intent to submit tissue for central HER2 testing
* Stage T1N1-2, T2-3N0-2, according to the American Joint Committee on Cancer (AJCC) 7th edition staging, based on the following minimum diagnostic work-up:

* Chest/abdominal/pelvic computed tomography (CT) or whole-body positron emission tomography (PET)/CT (NOTE: if CT is performed at this time point, whole-body PET/CT will be required prior to step 2 registration; PET/CT of skull base to mid-thigh is acceptable) (NOTE: if adenopathy is noted on CT or whole-body PET/CT scan, an endoscopic ultrasound is not required prior to STEP 2 registration as long as adequate tissue has been obtained for central HER2 testing)
* Patients may have regional adenopathy including para-esophageal, gastric, gastrohepatic and celiac nodes; if celiac adenopathy is present, it must be =\< 2 cm
* Patients with tumors at the level of the carina or above must undergo bronchoscopy to exclude fistula
* Zubrod performance status 0-2
* Absolute neutrophil count (ANC) \>= 1,500 cells/mm\^3
* Platelets \>= 100,000 cells/mm\^3
* Hemoglobin \>= 8.0 g/dL (note: the use of transfusion or other intervention to achieve hemoglobin \[Hgb\] \>= 8.0 g/dL is acceptable)
* Creatinine =\< 2 times upper limit of normal
* Bilirubin =\< 1.5 times upper limit of normal
* Aspartate aminotransferase (AST) =\< 3.0 times upper limit of normal
* For women of childbearing potential, a negative serum or urine pregnancy test
* Patients must sign a study-specific informed consent prior to study entry
* CONDITIONS FOR PATIENT ELIGIBILITY PRIOR TO STEP 2 REGISTRATION (HER2-POSITIVE PATIENTS ONLY)
* HER2 expressing adenocarcinoma of the esophagus centrally
* Surgical consultation to confirm that patient will be able to undergo curative resection after completion of chemoradiation within 56 days prior to step 2 registration
* Radiation oncology consultation to confirm that disease can be encompassed in a radiotherapy field within 56 days prior to step 2 registration
* Consultation with a medical oncologist within 56 days prior to step 2 registration
* Stage T1N1-2, T2-3N0-2, according to the AJCC 7th edition staging, based upon the following minimum diagnostic work-up:

* History/physical examination, with documentation of the patient's weight, within 14 days prior to step 2 registration
* Whole-body PET/CT scan within 56 days prior to step 2 registration (if only CT performed prior to step 1 registration)
* Endoscopic ultrasound within 56 days prior to step 2 registration, unless the patient is found to have adenopathy per CT or whole-body PET/CT scan
* Electrocardiogram (EKG) within 56 days prior to step 2 registration
* Serum creatinine =\< 2 x the upper limit or normal within 14 days prior to step 2 registration
* Zubrod performance status 0-2 within 14 days prior to step 2 registration
* For women of childbearing potential, a negative serum pregnancy test within 14 days prior to step 2 registration
* Left ventricular ejection fraction (LVEF) \>= institutional lower limit of normal by cardiac echocardiogram (echo) or multi gated acquisition (MUGA) scan within 56 days prior to step 2 registration
* Women of childbearing potential and sexually active male participants must agree to practice adequate contraception while on study and for at least 60 days following the last dose of chemotherapy or trastuzumab

Exclusion Criteria:

* Patients with cervical esophageal carcinoma
* Patients with T1N0 disease, T4 disease, and proximal esophageal cancers (15-24 cm)
* Prior systemic chemotherapy for esophageal cancer; note that prior chemotherapy for a different cancer is allowable
* Prior radiation therapy for esophageal cancer or prior chest radiotherapy
* Prior anthracycline or taxane
* Evidence of tracheoesophageal fistula or invasion into the trachea or major bronchi
* Prior invasive malignancy (except non-melanomatous skin cancer), unless disease free for a minimum of 2 years (e.g., carcinoma in situ of the breast, oral cavity, or cervix are permissible)
* Medical contraindications to esophagectomy
* Prior therapy with any agent targeting the HER2 pathway or human epidermal growth factor receptor 1 (HER1) (epidermal growth factor receptor \[EGFR\]) pathway
* Prior therapy with trastuzumab
* Prior allergic reaction to the study drugs involved in this protocol or to a monoclonal antibody
* Previous history of congestive heart failure
* Severe, active comorbidity, defined as follows:

* Unstable angina in the last 6 months
* Transmural myocardial infarction within the last 6 months
* Acute bacterial or fungal infection requiring intravenous antibiotics at the time of registration
* Acquired immune deficiency syndrome (AIDS) based upon current Centers for Disease Control and Prevention (CDC) definition; note, however that human immunodeficiency virus (HIV) testing is not required for entry into this protocol; protocol-specific requirements may also exclude immunocompromised patients
* Pregnant or nursing women or women of childbearing potential and men who are sexually active and not willing/able to use medically acceptable forms of contraception