The former president suffered sweeping losses in several swing races, reportedly blaming everyone who told him to endorse Dr. Oz.