Exhibit 5.1

LAW OFFICES

SILVER, FREEDMAN & TAFF, L.L.P.

A LIMITED LIABILITY PARTNERSHIP INCLUDING PROFESSIONAL CORPORATIONS

3299 K STREET, N.W., SUITE 100

WASHINGTON, D.C. 20007

PHONE: (202) 295-4500

FAX: (202) 337-5502

WWW.SFTLAW.COM

July 2, 2013

First PacTrust Bancorp, Inc.

18500 Von Karman Avenue, Suite 1100

Irvine, California 92612

Ladies and Gentlemen:

We have acted as special Maryland counsel to First PacTrust Bancorp, Inc., a Maryland corporation (the “Company”), in connection with the issuance and sale by the Company of an aggregate of 360,000 shares (the “Shares”) of the Company’s voting common stock, par value $0.01 per share, pursuant to the exercise of the overallotment option granted to the Underwriters (as defined below). The Shares are being sold pursuant to the Underwriting Agreement, dated as of June 21, 2013 (the “Underwriting Agreement”), by and between the Company and Raymond James & Associates, Inc., as representative of the several underwriters named in Schedule A thereto (collectively, the “Underwriters”). The Shares have been registered under the Securities Act of 1933, as amended, pursuant to a registration statement on Form S-3 filed by the Company with the Securities and Exchange Commission (the “Commission”), which became effective on November 23, 2010 (File No. 333-170622) (the “Registration Statement”), including a base prospectus dated November 23, 2010 (the “Base Prospectus”) and a prospectus supplement relating to the Shares dated June 21, 2013 (the “Prospectus Supplement,” and together with the Base Prospectus, the “Prospectus”).

In connection with our opinion, we have examined originals, or copies, certified or otherwise identified to our satisfaction, of the Underwriting Agreement, the Registration Statement, the Prospectus, the Company’s charter and bylaws, resolutions of the Company’s Board of Directors and committees thereof, certificates of public officials, certificates of corporate officers and such other documents and corporate records as we have deemed appropriate for the purpose of rendering this opinion. We have assumed without investigation the genuineness of all signatures, the legal capacity of natural persons, the authenticity, accuracy and completeness of all documents submitted to us as originals, the conformity to authentic and complete original documents of all documents submitted to us as certified, conformed or photostatic copies and the authenticity, accuracy and completeness of the originals of such copies. We have further assumed the due authorization of such documents by all parties other than the Company and the taking of all requisite action respecting such documents by all parties other than the Company, the due execution and delivery of such documents by each party other than the Company and have additionally assumed that all agreements are the valid and binding agreements of all parties to such agreements, other than the Company. In addition, we have assumed the accuracy of certifications of public officials, government agencies and departments, corporate officers, and individuals and statements of fact, on which we are relying, and have made no independent investigations thereof.

Based upon the foregoing, and subject to the limitations, qualifications, exceptions and assumptions set forth herein, we are of the opinion that the Shares have been duly authorized and, when issued and delivered as provided in the Underwriting Agreement and upon receipt by the Company of the consideration therefor as contemplated by the Underwriting Agreement, will be validly issued, fully paid and non-assessable.

First PacTrust Bancorp, Inc.

July 2, 2013

Page 2

In rendering the opinion set forth herein, we express no opinion as to the laws of any jurisdiction other than the General Corporation Law of the State of Maryland, as currently in effect. This opinion is limited to the facts bearing on this opinion as they exist on the date of this letter. We disclaim any obligation to review or supplement this opinion or to advise you of any changes in the circumstances, laws or events that may occur after this date or otherwise update this opinion.

We hereby consent to the filing of this opinion as an exhibit to the Company’s Current Report on Form 8-K filed on July 2, 2013, and to the reference to our name under the heading “Legal Matters” in the Prospectus. In giving our consent, we do not thereby admit that we are in the category of persons whose consent is required under Section 7 of the Securities Act or the rules and regulations of the Commission thereunder. The opinions expressed herein are matters of professional judgment and are not a guarantee of result.