* STEP I INITIAL REGISTRATION: BRAFV600E TESTING:
* Patients must have histologically or cytologically documented adenocarcinoma of the colon or rectum that is either metastatic, or locally advanced and unresectable
* Patients must have BRAFV600E mutant status documented by a Clinical Laboratory Improvements Amendments (CLIA) certified laboratory on a pathology report prior to Step 2 registration; use of an Food and Drug Administration (FDA)-approved test is preferred although other BRAF tests at a CLIA-certified laboratory may also be accepted; if a BRAFV600E mutation is known, then the patient must be registered to Step 2 Randomization immediately following Step 1 Initial Registration; if testing has not been performed locally, BRAFV600E testing must be completed by the central lab prior to Step 2 Randomization; if the specimen does not have a BRAFV600E mutation, the patient is ineligible for Step 2 Randomization
* Brain metastases are allowed if they have been adequately treated with radiotherapy or surgery and stable for at least 90 days prior to Step 1 Initial Registration; eligible patients should be neurologically asymptomatic and without corticosteroid treatment for at least 7 days prior to Step 1 Initial Registration
* Patients must have had one or two prior regimens of systemic chemotherapy for metastatic disease; prior treatment with irinotecan is allowed; a maintenance regimen of 5-fluorouracil or capecitabine, with or without bevacizumab, should not be counted as a separate line of treatment; prior treatment for metastatic disease is not required for patients who experienced disease recurrence during or within 6 months of completion of adjuvant chemotherapy
* Patients must not have been treated with any of the following prior to Step 2 Randomization:

* Cetuximab, panitumumab, or any other monoclonal antibody against EGFR or inhibitor of EGFR
* BRAF inhibitor including, but not limited to, vemurafenib or dabrafenib; regorafenib is not considered a BRAF inhibitor for the purpose of determining trial eligibility
* Mitogen-activated protein/extracellular signal-regulated kinase (MEK) inhibitor including, but not limited to, trametinib or selumetinib
* Previous chemotherapy, immunotherapy, or radiation therapy must have been completed at least 14 days prior to Step 1 Initial Registration and all toxicity must be resolved to Common Terminology Criteria for Adverse Events version 4.0 (CTCAE v4.0) grade 1 (with the exception of CTCAE v4.0 grade 2 neuropathy) prior to Step 1 Initial Registration
* Patients must not have a tumor with a mutation detected in codons 12 or 13 in KRAS; patients must not have a tumor with a known mutation detected in codons 61, 117, or 146 of KRAS or NRAS
* SPECIMEN SUBMISSION CRITERIA:
* Patients must have tumor (slides or block) available for submission for V600E BRAF testing
* Patients must have additional tumor available and be willing to submit tissue and blood samples
* SPECIMEN SUBMISSION CRITERIA REGULATORY CRITERIA:
* Patients or their legally authorized representative must be informed of the investigational nature of this study and must sign and give written informed consent in accordance with institutional and federal guidelines; for Step 1 Initial Registration of patients who have not yet submitted specimens for the central BRAFV600E testing, the appropriate consent form is the Step 1 Consent Form; for both Step 1 Initial Registration and Step 2 Randomization of patients whose BRAF mutation status is already known, the appropriate consent form is the Step 2 Consent Form
* STEP 2 RANDOMIZATION:
* Patients must have BRAFV600E mutation
* Patients must have measurable or non-measurable metastatic disease; computed tomography (CT) scans or magnetic resonance imaging (MRIs) used to assess all disease must have been completed within 28 days prior to Step 2 Randomization; CT scans or MRIs must be assessed and documented on the Baseline Tumor Assessment Form (Response Evaluation Criteria in Solid Tumors \[RECIST\] 1.1)
* Patients must have a Zubrod performance status of 0-1
* Absolute neutrophil count (ANC) \>= 1,500/mcL
* Platelets \>= 100,00/mcL
* Hemoglobin \>= 9 g/dL
* Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) =\< 2.5 x institutional upper limit of normal (IULN) or =\< 5 x IULN if liver metastases are present
* Total bilirubin =\< 1.5 x IULN
* Serum creatinine =\< 1.5 x IULN within 14 days prior to Step 2 Randomization OR
* Calculated creatinine clearance \> 60 ml/min; the serum creatinine value used in the calculation must have been obtained within 14 days prior to Step 2 Randomization
* Patients must have an electrocardiogram (ECG) within 14 days prior to Step 2 Randomization
* Patients must have corrected QT (QTc) =\< 500 msec
* Patients must not have a known history of Gilbert's Syndrome or known homozygosity for the UDP glucuronosyltransferase 1 family, polypeptide A1 (UGT1A1)\*28 allele
* Patients must not have interstitial pneumonia or extensive symptomatic interstitial fibrosis of the lung
* Patients must not have an uncontrolled intercurrent illness including, but not limited to, active bleeding diathesis, uncontrolled infection/disorders, nonmalignant medical illnesses that are uncontrolled or whose control may be jeopardized by the treatment with the study therapy, or psychiatric illness/social situations which would limit compliance with study requirements
* Patients must be able to swallow pill/tablet and have no refractory nausea, vomiting, malabsorption, external biliary shunt, or significant small bowel resection that would preclude adequate absorption
* Patients must not be pregnant or nursing; women/men of reproductive potential must have agreed to use an effective contraceptive method while on study and for 30 days after study treatment; a woman is considered to be of "reproductive potential" if she has had menses at any time in the preceding 12 consecutive months; in addition to routine contraceptive methods, "effective contraception" also includes heterosexual celibacy and surgery intended to prevent pregnancy (or with a side-effect of pregnancy prevention) defined as a hysterectomy, bilateral oophorectomy or bilateral tubal ligation; however, if at any point a previously celibate patient chooses to become heterosexually active during the time period for use of contraceptive measures, he/she is responsible for beginning contraceptive measures
* No other prior malignancy is allowed except for the following: adequately treated basal cell or squamous cell skin cancer, in situ cervical cancer, adequately treated stage I or II cancer from which the patient is currently in complete remission, or any other cancer from which the patient has been disease free for three years
* STEP 2 RANDOMIZATION REGULATORY CRITERIA:
* Patients or their legally authorized representative must be informed of the investigational nature of this study and must sign and give written informed consent in accordance with institutional and federal guidelines; for all patients, the appropriate consent form for this registration is the Step 2 Consent Form
* As a part of the OPEN registration process the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system
* STEP 3 CROSSOVER REGISTRATION:
* Patients must have documented disease progression while on Arm 1 of this protocol; the follow-up tumor assessment form documenting disease progression must be submitted to Southwestern Oncology Group (SWOG) prior to Step 3
* Registration to Step 3 Crossover must be within 28 days of discontinuation of Arm 1 protocol treatment; patients going off treatment for any other reason are not eligible
* ANC \>= 1,500/mcL within 14 days prior to Step 3 registration
* Platelets \>= 100,00/mcL within 14 days prior to Step 3 registration
* Hemoglobin \>= 9 g/dL within 14 days prior to Step 3 registration
* AST and ALT =\< 2.5 x institutional upper limit of normal (IULN) or =\< 5 x IULN if liver metastases are present within 14 days prior to Step 3 registration
* Total bilirubin =\< 1.5 x IULN within 14 days prior to Step 3 registration
* Serum creatinine =\< 1.5 x IULN within 14 days prior to Step 3 registration OR
* Calculated creatinine clearance \> 60 ml/min; the serum creatinine value used in the calculation must have been obtained within 14 days prior to Step 3 registration