EXHIBIT 23

Consent of Independent Registered Public Accounting Firm

The Board of Directors and Stockholders

Heritage Financial Corporation:

We consent to the incorporation by reference in the registration statements (No. 333-57513, No. 333-71415, No. 333-87599, No. 333-88976, No. 333-88980, No. 333-88982, No. 333-134473, No. 333-134474 and No. 333-134475) on Form S-8 and the registration statement (No. 333-156271) on Form S-3 of Heritage Financial Corporation of our reports dated March 2, 2010, with respect to the consolidated statements of financial condition of Heritage Financial Corporation and subsidiaries as of December 31, 2009 and 2008, and the related consolidated statements of operations, stockholders’ equity and comprehensive income (loss) and cash flows for each of the years in the three-year period ended December 31, 2009, and the effectiveness of internal control over financial reporting as of December 31, 2009, which reports appear in the December 31, 2009 annual report on Form 10-K of Heritage Financial Corporation.

/s/ KPMG, LLP

Seattle, Washington

March 2, 2010