The outlook for office construction in Manhattan is getting darker still. A forecast from the New York Building Congress on Wednesday predicts that the rate of building activity in the borough will stagnate, with around 2 million square feet being added annually through the end of the decade.

This year the group, which is made up of firms in the design, construction and real estate industries, anticipates 2.1 million square feet of offices will be built below 60th Street, a figure that is expected to slump to 1.7 million square feet next year—putting it roughly on par with the amount of space added in 2011.

“We need a stronger performance globally and nationally,” said Richard Anderson, president of the Building Congress. “People don’t put up office buildings without more leasing, assurances and bank financing.”

A lack of demand for space has prevented greater office construction. Roughly 47% less office space was leased in the second quarter of 2012 than the second quarter of the previous year, according to a report from real estate brokerage Cushman & Wakefield Inc. on Tuesday.

One of the reasons is that companies are squeezing more workers into the same amount of space. Office tenants historically used 250 square feet for every worker, on average, rather than the 200 allotted today, according to the Cushman report. Also, there is still plenty of vacant space to go around, with an overall vacancy rate in Manhattan of 9.1% as of the second quarter.

While jobs are being added, the market needs more to spur additional construction. This year, 38,000 office jobs have already been added, compared with 36,000 for all of 2011. But employment remains below its peak in 2008 of almost 1.8 million jobs.

“More than anything else, we need to grow office jobs,” Mr. Anderson said. “We’ve been growing, but it’s been filling up vacant space.”

The Building Congress estimates that this could be the third consecutive decade in which fewer than 20 million square feet of offices will be built. Office construction in the four decades after World War II averaged nearly 5 million square feet annually, according the Building Congress.

But that may not be all bad news for the office market. Rapid building in the past, particularly in the 1980s, was fueled by a larger proportion of spec building, not demand, said Barbara Denham, senior economist for Eastern Consolidated. Many of those buildings sat vacant and pushed rents down, she said.

“It turned out to be a really bad thing for New York’s real estate markets,” Ms. Denham said. “There’s a lot of global uncertainty, but the market should be smarter than it was. Overbuilding is never to anyone’s advantage