The settlement, which Goldman disclosed in January, stems from the firm's conduct in packaging, securitization, marketing and sale of residential mortgage-backed securities between 2005 and 2007, the Justice Department said.
Investors suffered billions of dollars in losses from the securities bought during the period, the department said.
The settlement comprises a $2.385 billion civil penalty and $1.8 billion in other relief, including funds for homeowners whose mortgages exceed the value of their property, as well as distressed borrowers. It also preserves the government's ability to bring criminal charges against Goldman and does not release any individuals from potential criminal or civil liability, the Justice Department said.
In addition, Goldman will pay $875 million to resolve claims by the New York and Illinois attorneys general, the National Credit Union Administration and the Federal Home Loan Banks of Chicago and Seattle.
A state and federal working group formed to investigate wrongdoing in the pre-financial crisis mortgage-backed securities market negotiated the settlement, said New York Attorney General Eric Schneiderman.
The group has reached settlements with five other major financial institutions since 2012: J.P. Morgan Chase (JPM.N) ($13 billion), Bank of America (BAC.N) ($16.6 billion), Citibank (C.N) ($7 billion) and Morgan Stanley (MS.N) ($3.2 billion).
"We are pleased to put these legacy matters behind us," a Goldman spokesman said in a statement. "Since the financial crisis, we have taken significant steps to strengthen our culture, reinforce our commitment to our clients, and ensure our governance processes are robust," he said.
Goldman also acknowledged a Justice Department statement of facts describing how the firm misled investors.
For example, Goldman's due diligence for one issue of 2006 mortgage-backed securities showed that some of the loan pools reflected an “unusually high” percentage of loans with credit and compliance programs, the Department said.
"How do we know that we caught everything?" asked a Goldman committee tasked with reviewing and approving mortgage-backed securities, according to the Justice Department. "We don't," a Goldman manager said.
"Depends on what you mean by everything? Because of the limited sampling... we don’t catch everything,” another Goldman manager said.
Still, the committee approved the securities without requiring additional due diligence, said the Justice Department, which did not identify those involved.