Section 2 — Exhibit 99.1

[FOR IMMEDIATE RELEASE]

First Business Financial Services, Inc.

401 Charmany Drive

Madison, WI 53719

FIRST BUSINESS FINANCIAL SERVICES, INC. ANNOUNCES RECORD 2011 EARNINGS

Madison, WI – March 9, 2012 (GLOBE NEWSWIRE) – First Business Financial Services, Inc. (the “Company”) (NASDAQ: FBIZ), the parent company of First Business Bank and First Business Bank – Milwaukee, today reported strong fourth quarter and record full year 2011 net income, as the Company continued to increase top line revenue, reduce the level of non-performing assets and improve operating efficiency.

Highlights for the periods ended December 31, 2011 include:

The Company recorded fourth quarter 2011 net income of $2.4 million, or diluted earnings per common share of $0.90, compared to net income of $598,000, or diluted earnings per common share of $0.24, for the fourth quarter of 2010.

The Company increased net income for the year ended December 31, 2011 to a record $8.4 million, or $3.23 per diluted common share. This compares to net income of $941,000, or $0.37 per diluted common share earned in the year ended 2010. Adjusted to exclude the impact of the $2.7 million goodwill impairment charge in the second quarter of 2010, 2011 net income grew $4.8 million as compared to the prior year.

“In 2011 we executed on our strategic goals, delivering record earnings and significant growth in top line revenue and in-market deposits,” said Corey A. Chambas, President and Chief Executive Officer. “We achieved important efficiency gains and meaningful improvement in asset quality. We are confident our consistent and proven approach to building value has placed First Business in a very strong position for continued success in 2012.”

Net interest income increased 7.9% to $8.9 million in the fourth quarter of 2011, compared to $8.3 million for the fourth quarter of 2010. The improvement reflects a 15 basis point widening of the net interest margin to 3.23%, primarily resulting from a 42 basis point decline in the average rate paid on interest-bearing deposit balances. The decrease in the overall rate on the interest-bearing funds was primarily caused by the replacement of maturing certificates of deposits, including brokered certificates of deposits, at lower current market rates and a lower rate paid on money market accounts. Consequently, interest expense for the fourth quarter decreased 16.0% to $5.0 million, compared to $5.9 million for the fourth quarter 2010.

Net interest income for the year ended December 31, 2011 was $35.5 million, an increase of $3.5 million or 11.0% compared to the prior year. Interest income of $56.2 million for the year ended December 31, 2011 was relatively flat when compared to prior year, while interest expense of $20.8 million decreased $3.9 million, or 15.9% compared to 2010. The primary driver of the decline in interest expense was a 39 basis point decline in the average rate paid on interest-bearing deposits, resulting in a 25 basis point increase in the net interest margin to 3.29% for the year ended December 31, 2011 as compared to 3.04% for the prior year. The Federal Reserve held interest rates constant throughout 2010 and 2011, which facilitated pricing for new deposits and repricing of maturing deposits commensurate with current market conditions and demand, along with the replacement of higher rate maturing brokered certificates of deposits at lower current market rates.

Non-interest income for the fourth quarter of 2011 was $1.9 million, an increase of $158,000, or 9.0%, compared with the same period of 2010. Service charges on deposits grew 13.7%, or $60,000, and increased asset-based lending activity drove growth in loan fees of $51,000, or 14.5%, compared to the fourth quarter of 2010.

Non-interest income for the year ended December 31, 2011 totaled $7.1 million, which was $317,000, or 4.7%, higher than fees in 2010. Fee income generated by the Trust and Investment Services business increased by $198,000, or 8.5%, to $2.5 million for the year ended December 31, 2011. The increase in revenue is related to 26% growth in assets under management and administration at year end, as compared to a year ago. In addition, loan fees grew by $243,000 or 19.6% compared to the year ended December 31, 2010, as increased asset based lending activity resulted in higher collateral audit fees. Lower volume and gains on lease end terminations drove a partially offsetting decline of $181,000, or 29.3%, in other income.

Non-interest expense for the fourth quarter of 2011 was $6.2 million, virtually flat compared to the same quarter in 2010. Compensation costs of $3.5 million were $530,000, or 17.9%, higher than the fourth quarter of 2010 due to a larger accrual to fund the Company’s non-equity incentive compensation program. The higher accrual in 2011 reflects favorable 2011 financial performance. Significant declines in FDIC insurance costs and collateral liquidation costs helped offset expense growth. FDIC insurance expense declined $216,000, or 27.0%, from the fourth quarter of 2010, and collateral liquidation costs fell by $111,000, or 34.4%.

Non-interest expense for the year ended December 31, 2011 totaled $26.4 million, which was $2.0 million lower than in 2010. Excluding the $2.7 million goodwill impairment charge recognized in the second quarter of 2010, 2011 non-interest expenses increased $726,000, or 2.8%. Modest expense growth for the year ended December 31, 2011 was primarily related to increased costs to fund incentive compensation accruals in line with favorable 2011 financial performance. Partially offsetting core expense growth was the favorable change in FDIC insurance costs, which drove a decline of $644,000, or 20.6%, compared to the prior year. In addition, collateral liquidation costs declined by $381,000, or 32.6%, reflecting successful outcomes of the Company’s efforts to minimize impaired loan resolution costs. Coupled with top line revenue growth of 9.9%, 2011 cost containment drove an improvement in the efficiency ratio to 61.0%, which was 474 basis points lower than in the prior year period.

The provision for loan and lease losses for the fourth quarter of 2011 was $937,000, representing a decline of $1.7 million or 65.0% compared to the same quarter of the prior year. The provision for loan and lease losses for the year ended December 31, 2011 decreased 39.7% to $4.3 million from $7.0 million for the prior year.

Asset quality improved throughout 2011. The ratio of nonperforming assets to total assets fell 159 basis points from 3.63% at December 31, 2010 to 2.04% at December 31, 2011. Non-accrual loans and leases decreased by 43.3%, or $16.6 million, during the year ended December 31, 2011, reflecting payoffs, paydowns and charge-offs, which were partially offset by continued additions of newly identified problem loans and leases.

Total assets grew $70.1 million to $1.2 billion as of December 31, 2011, which represents growth of 6.3% from December 31, 2010. Cash and cash equivalents increased $79.3 million and available-for-sale securities grew $17.0 million as the Company attempts to provide an enhanced return on excess liquidity. Growth in short term investments and securities was partially offset by a $24.2 million reduction in net loan and lease balances since December 31, 2010. This 2.8% decline in net loans and leases is attributable to new loan originations being offset by amortization of the existing loan portfolio and the reduction of non-accrual loans and leases.

Dividend Maintained

During the fourth quarter of 2011 the Company’s Board of Directors approved a $0.07 quarterly cash dividend on its common stock, which was paid on January 15, 2012 to shareholders of record at the close of business on January 1, 2012. This maintained the Company’s annualized dividend at $0.28 per share, a level it has maintained for sixteen consecutive quarters.

Capital Strength

The Company’s capital ratios continue to improve and are in excess of the highest required regulatory benchmark levels. Total capital to risk-weighted assets was 13.1% as of December 31, 2011 as compared to 11.2% at December 31, 2010.

About First Business Financial Services, Inc.

First Business Financial Services (NASDAQ: FBIZ) is a $1.2 billion Wisconsin-based bank holding company that specializes in focused financial solutions for businesses, key executives, and high net worth individuals through its operating companies. It is the second largest Wisconsin-based commercial bank holding company listed on NASDAQ or New York Stock Exchange. Its companies include: First Business Bank—Madison; First Business Bank—Milwaukee; First Business Bank—Northeast; First Business Trust & Investments; First Business Equipment Finance, LLC; and First Business Capital Corp. For additional information, visit www.firstbusiness.com or call (608) 238-8008.

This press release includes “forward-looking” statements related to First Business Financial Services, Inc. (the “Company”) that can generally be identified as describing the Company’s future plans, objectives or goals. Such forward-looking statements are subject to risks and uncertainties that could cause actual results or outcomes to differ materially from those currently anticipated. These forward-looking statements are made pursuant to the safe harbor provisions of the Private Securities Litigation Reform Act of 1995. For further information about the factors that could affect the Company’s future results, please see the Company’s annual report on Form 10-K and other filings with the Securities and Exchange Commission.

James F. Ropella, Senior Vice President

and Chief Financial Officer

608-232-5970

jropella@firstbusiness.com

Selected Financial Information (Unaudited)

SELECTED FINANCIAL CONDITION DATA

STATEMENTS OF INCOME

SELECTED FINANCIAL RATIOS

ASSET QUALITY RATIOS

NON-GAAP RECONCILIATIONS

CORE EARNINGS