Even as trading volumes handled by these relatively small futures commission merchants (FCM) boomed over the past decade, profits were dwindling: electronic trading, the rise of the hedge fund and rapid-fire algorithmic trader, and the slump in interest rates had upended their century-old business model.

Some turned to speculating with their own money, trade finance or market research to bolster earnings; others have been sold, further depleting the ranks of an industry which saw market share halved in 10 years. Two, MF Global and Peregrine, allegedly looted their customers' money to try to stay afloat.

"The root cause of issues at both firms was the lack of profitability," says Gary DeWaal, general counsel of broker Newedge, which is owned by two French banks and is one of the world's largest futures brokerages.

"That caused the principals to do things that, in the end, were probably not such a good idea," he deadpanned.

The worst may not be over. Every new proposal to restore traders' trust in the futures markets threatens to further erode already wafer-thin margins. And an obscure new swaps rule imposed as part of the Dodd-Frank financial overhaul is amping up the risk brokers must take to hold their clients' cash.

Within that context, it becomes somewhat easier to see why Peregrine's founder and chief Russell Wasendorf Sr., who cast himself as a leading figure in the futures world until last week, said in his confession he "hated" the industry. Beneath the trappings of success, including a private jet and an $18 million headquarters, was a two-decade fraud in which he used over $200 million of customer funds to stave off collapse.

Jerry Markham, a law professor at Florida International University in Miami who has written extensively about regulations and collateral, summed up the challenge ahead:

"We've got to fix and protect those funds in a way that protects the customers, and at the same time lets the futures commission merchants earn money on those funds."

GENERATIONS

Often family-run operations stretching back generations, the independently owned brokers have a longstanding tradition on the trading floor, one that -- until last October -- included a nearly unblemished record of safeguarding customer funds. R.J. O'Brien, now the biggest "boutique" firm, dates back to 1914.

Increasingly they rely on retail traders, small companies or farmers that big banks won't touch -- clients who demand lots of service, but generate relatively few trading fees in return.

While big banks like JP Morgan (JPM.N) and Goldman Sachs (GS.N) can use their vast balance sheets, longstanding business ties or financing arrangements to support the brokerage business, small brokers have no such advantage.

"In this environment, the viability of independent FCMs is in question, surely," says Craig Pirrong, a professor of finance at the University of Houston.

The rise of screen-based trading cut down the need for floor brokers. Then the rise of high-frequency traders and big hedge funds drove even more trade toward the big banks equipped to handle the flow. In the past few years, the extension of near-zero interest rates for years eradicated hope of a rebound in a key source of income: interest on customers' margin.

While most large brokers must pass interest earned on collateral back to their customers, smaller firms were able to retain most of that revenue.

As Wasendorf siphoned millions from his own customers accounts, PFGBest, as his firm was known, struggled. The company had a net income of $920,000 in 2010, according to an earnings statement seen by Reuters. In 2003, the only other year for which data is available, it lost $500,000.

Some rivals in the business who have been contacted by PFGBest's customers say they were shocked by what they heard: PFGBest's trading margins were among the thinnest in the industry.

SHRINKING

While investment in U.S. futures markets has trebled over the past decade, the independent brokers' volume of so-called "segregated funds" -- collateral held against their customers' trading positions -- has barely risen.

Including Peregrine, independent brokers held segregated funds totaling about $13.4 billion as of May, about 9.5 percent of total U.S. seg funds, according to a Reuters analysis of Commodity Futures Trading Commission (CFTC) data.

A year ago, just before small investment bank Jefferies Group Inc (JEF.N) bought large independent broker Prudential Bache, it was 15 percent; ten years ago more than 20 percent.

Meanwhile the concentration of holdings among the upper ranks of broker-dealers has grown. In June 2002, the top 10 brokers -- including one independent firm, Refco, which later collapsed -- held 62 percent of total segregated funds; in May this year, they held 78 percent, all of them major banks.

"I'm not sure that level of concentration is good," says DeWaal, his own firm the third-largest on the list.

Some have simply run out of steam. Dallas-based Penson Worldwide agreed in May to sell its brokerage division to Knight Capital Group, an equities market-maker, for just $5 million after struggling to renegotiate its heavy debt load and staunch losses that totaled more than $225 million over six months.

Revenues at INTL FCStone Inc (INTL.O) have quadrupled over the past four years through a series of modest acquisitions, including MF Global's metals trading division, but it has struggled to remain profitable in recent months. Second-quarter net income fell 84 percent to just $2.4 million.

HARD TO BUILD, EASY TO BREAK

Rosenthal Collins Group (RCG), the second-largest independent brokerage firm, was among the brokers who moved quickly last week to reassure customers of the safety of their collateral. It began publishing a breakdown of where it keeps customer funds on a monthly basis after MF Global; now it does so daily.

"Everybody gets tarred with the brush," says Les Rosenthal, managing partner at RCG.

The industry debate over better ways to protect customer money has intensified. The CME Group last week openly questioned whether brokers should be allowed to continue being allowed to hold client collateral; some have suggested a third-party depository could work. Even if a system is established to allow interest income to flow back to the brokers, it adds cost.

Meanwhile some new measures bearing down on the brokers that date back to the financial crisis in 2008. The reforms that are driving the vast over-the-counter derivatives market onto exchanges offers some hope for new trading opportunities, but also complex -- and potentially costly -- new regulations.

One is a change in the way that exchanges collect collateral from the brokers who place customers' trade. In the past, a number of exchanges allowed so-called "net" margining. Under that system, a broker is required to transfer only enough funds to meet the sum total of its customers' positions.

If one customer was long and another was short the same product, the margin requirements effectively canceled each other out -- allowing the broker to retain more of their customers' escrow and collect interest income on those sums.

But under the Dodd-Frank financial reforms, clearing houses by November 8 must use a "gross margin", in which brokers must ensure each individual client's trading position is fully margined at the exchange level -- leaving the broker less cash.

"After November 8, we will go to a true gross model for all cleared business so that the amounts collected by the FCM and the clearinghouse will be identical," said CME Group (CME.O) spokesman Damon Leavell of the CME's Chicago Board of Trade.

A second issue looms as well: the move to "legally separate, operationally commingled" collateral.

In futures trading, all client money at a given broker goes into a single pot, so if one client goes under and pulls the broker with it, other clients may have to share in losses.

Under the new rules, which currently apply only to swaps, brokers will still put all the money in one pot, but will need to keep the assets of each customer legally siloed from the assets of every other customer, so that if one client fails, the other customers will still be able to get all their money out, leaving other brokers to step in if the one with the defaulting customer goes under.

"We've replaced fellow-customer risk with fellow-broker risk," says DeWaal, wondering aloud how much more pressure the industry can bear: "At some point people have to wonder if opening up a hamburger stand would be better."

(Additional reporting by Jonathan Leff; Editing by Alden Bentley)

• A Permanent Economic Emergency