Investigators found a suicide note in his office, which was parked on one lane of the bridge linking the city suburb of San Pedro to Terminal Island, the Los Angeles Times reported. Its content has not been revealed.
The newspaper earlier reported a suicide note had been found in his car.
A witness saw Scott, 68, climb over a fence on the bridge and jump into the water, according to Lieutenant Joseph Bale of the city coroner's office. Other witnesses said they later saw Scott's shoes floating in the water.
Authorities summoned divers and a helicopter in a bid to find Scott's body in the port's murky waters. It was recovered at approximately 3:00 pm (2200 GMT), four hours after he jumped, and turned over to coroner officials, Bale said.
The search has delayed a ferry linking the coast with Santa Catalina island.
Scott was the younger brother of fellow film director Ridley Scott, the maker of Oscar-winning movies like "Gladiator" and "Black Hawk Down" as well as other hits like "Blade Runner."
The family has confirmed Scott's death, but offered no details.
God's mercy on the man, God's comfort to his family.