Exhibit 10.164

EXECUTION

AMENDMENT NO. 11

TO MORTGAGE LOAN PARTICIPATION PURCHASE AND SALE AGREEMENT

Amendment No. 11 to Mortgage Loan Participation Purchase And Sale Agreement, dated as of March 29, 2016 (this “Amendment”), by and among Bank of America, N.A. (“Purchaser”), PennyMac Corp. (“Seller”), PennyMac Mortgage Investment Trust and PennyMac Operating Partnership, L.P. (individually and collectively, the “Guarantor”).

RECITALS

Purchaser, Guarantor and Seller are parties to that certain Mortgage Loan Participation Purchase And Sale Agreement, dated as of December 23, 2011 (as amended, restated, supplemented or otherwise modified from time to time, the “Existing MLPSA”; and as further amended by this Amendment, the “MLPSA”). The Guarantor is a party to that certain Guaranty (as amended, restated, supplemented or otherwise modified from time to time, the “Guaranty”), dated as of December 23, 2011, made by Guarantor in favor of Purchaser.

Purchaser, Seller and Guarantor have agreed, subject to the terms and conditions of this Amendment, that the Existing MLPSA be amended to reflect certain agreed upon revisions to the terms of the Existing MLPSA. As a condition precedent to amending the Existing MLPSA, Purchaser has required Guarantor to ratify and affirm the Guaranty on the date hereof.

Accordingly, Purchaser, Seller and Guarantor hereby agree, in consideration of the mutual promises and mutual obligations set forth herein, that the Existing MLPSA is hereby amended as follows:

Section 1. Definitions. Section 1 of the Existing MLPSA is hereby amended by:

1.1 deleting the definitions of “Discount Rate”, “Expiration Date”, “LIBOR” and “Security Issuance Failure” in its entirety and replacing it with the following:

“Discount Rate”: With respect to each Participation Certificate, a discount rate determined as of the related Purchase Date equal to (a) the greater of (i) LIBOR, and (ii) the LIBOR Floor, plus (b) the Applicable Percentage. Notwithstanding the foregoing, under no circumstances shall the Discount Rate be less than zero.

“Expiration Date”: The earlier of (i) March 28, 2017, (ii) at Purchaser’s option, upon the occurrence of an Event of Default, and (iii) the date on which this Agreement shall terminate in accordance with the provisions hereof or by operation of law.

“LIBOR”: The daily rate per annum (rounded to four (4) decimal places) for one-month U.S. dollar denominated deposits as offered to prime banks in the London interbank market, as published on the Official ICE LIBOR Fixings page by Bloomberg or in the Wall Street Journal as of the date of determination; provided, that if Purchaser determines that any law, regulation, treaty or directive or any change therein or in the interpretation or application thereof, or any circumstance materially and adversely

affecting the London interbank market, shall make it unlawful, impractical or commercially unreasonable for Purchaser to purchase Participation Certificates as contemplated by this Agreement using LIBOR, then Purchaser may select an alternative rate of interest or index in its discretion.

“Security Issuance Failure”: The Failure of the Security (a) to be issued for any reason within the reasonable control of the Seller (as determined by Purchaser in its sole good faith discretion), including but not limited to Seller’s failure to perform any of its obligations under this Agreement or any other Program Document or failure to perform in Strict Compliance with the related Agency Program, (b) to be issued for any reason outside of the reasonable control of the Seller (as determined by Purchaser in its sole good faith discretion), including, but not limited to, third party systems failures, or (c) to be Delivered to Purchaser or its designee (such designee being properly notified it is holding such Security for Purchaser); provided, that solely with respect to clauses (b) and (c) a Security Issuance Failure shall not have occurred to the extent (i) Seller has performed its obligations under this Agreement and each other Program Document; (ii) Seller has performed in Strict Compliance with the related Agency Program; (iii) such failure to be issued or Delivered arises solely from the acts or omissions of a party other than the Seller (as determined by Purchaser in its sole good faith discretion) and (iv) such failure is cured within one (1) Business Day of Seller’s notice or knowledge of such failure.

1.2 adding the following definition in its proper alphabetical order:

“LIBOR Floor”: 0%.

Section 2. Events of Default. Section 6(g) of the Existing MLPSA is hereby amended by adding the following new subclause:

(xxi) Seller has, without the express written consent of Purchaser, entered into any settlement with, or consented to the issuance of a consent order by, any Governmental Authority in which the fines, penalties, settlement amounts or any other amounts owed by Seller thereunder exceeds $5,000,000 in the aggregate.

Section 3. Fees and Expenses. Seller hereby agrees to pay to Purchaser, on demand, any and all reasonable fees, costs and expenses (including reasonable fees and expenses of counsel) incurred by Purchaser in connection with the development, preparation and execution of this Amendment, irrespective of whether any transactions hereunder are executed.

Section 4. Conditions Precedent. This Amendment shall become effective as of the date hereof (the “Amendment Effective Date”), subject to the satisfaction of the following conditions precedent:

4.1 Delivered Documents. On the Amendment Effective Date, the Purchaser shall have received this Amendment, executed and delivered by a duly authorized officer of Purchaser, Seller and Guarantor.

2

4.2 Facility Fee. Seller shall have paid to Purchaser in immediately available funds that portion of the Facility Fee due and payable on the Amendment Effective Date.

Section 5. Limited Effect. Except as expressly amended and modified by this Amendment, the Existing MLPSA shall continue to be, and shall remain, in full force and effect in accordance with its terms.

Section 6. Counterparts. This Amendment may be executed by each of the parties hereto on any number of separate counterparts, each of which shall be an original and all of which taken together shall constitute one and the same instrument.

Section 7. Severability. Each provision and agreement herein shall be treated as separate and independent from any other provision or agreement herein and shall be enforceable notwithstanding the unenforceability of any such other provision or agreement.

SECTION 8. GOVERNING LAW. THE AMENDMENT AND THE RIGHTS AND OBLIGATIONS OF THE PARTIES HEREUNDER SHALL BE CONSTRUED IN ACCORDANCE WITH, AND GOVERNED BY, THE LAW OF THE STATE OF NEW YORK, WITHOUT GIVING EFFECT TO THE CONFLICT OF LAWS PRINCIPLES THEREOF.

Section 9. Reaffirmation of Guaranty. The Guarantor hereby (i) agrees that the liability of Guarantor or rights of Purchaser under the Guaranty shall not be affected as a result of this Amendment, (ii) ratifies and affirms all of the terms, covenants, conditions and obligations of the Guaranty and (iii) acknowledges and agrees that such Guaranty is and shall continue to be in full force and effect.

[SIGNATURE PAGE FOLLOWS]

3

IN WITNESS WHEREOF, the parties have caused their names to be signed hereto by their respective officers thereunto duly authorized as of the day and year first above written.

Signature Page to Amendment No. 11 to MLPSA