Exhibit 10.4

SUBORDINATED DEBT SETTLEMENT AND PREFERRED STOCK

REPURCHASE AGREEMENT

THIS SUBORDINATED DEBT SETTLEMENT AND PREFERRED STOCK REPURCHASE AGREEMENT (this “Agreement”) is dated as of August 1, 2011 by and between SunTrust Bank, a Georgia banking corporation (“SunTrust”), and CommunityOne Bank, N.A., a national banking association (“Bank”).

WHEREAS, as of the date hereof, SunTrust is the holder and the Bank is the maker of a Second Amended and Restated Subordinated Note Due 2015, the principal amount of which is $2,500,000 (“Note”);

WHEREAS, as of the date hereof, SunTrust owns 12,500,000 shares of the Bank’s non-voting, non-convertible, non-redeemable cumulative preferred stock with an aggregate liquidation preference of $12,500,000 (“Preferred Stock”);

WHEREAS, the Bank’s parent holding company, FNB United Corp. (“FNB”), has entered into investment agreements with affiliates of The Carlyle Group and Oak Hill Capital Partners on April 26, 2011 for the purchase and sale of FNB’s common stock in the aggregate amount of $155 million (“Investment Agreements”) as part of FNB’s effort to raise $310 million in common equity capital;

WHEREAS, as part of the overall recapitalization, FNB has entered into an Agreement and Plan of Merger with Bank of Granite Corporation (“Granite”) on April 26, 2011 (“Merger Agreement”) for FNB to acquire Granite through the merger of a wholly owned subsidiary of FNB established solely for the purposes of the merger with and into Granite;

WHEREAS, the settlement of the Note and the repurchase of the Preferred Stock is a condition to the closing of both the Investment Agreements and the Merger Agreement (the “Primary Transactions”);

WHEREAS, both the Bank and SunTrust believe that settlement of the Note and the repurchase of the Stock is in the best interests of both parties.

NOW, THEREFORE, for good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the parties hereto do hereby agree as follows:

1. Subject to the conditions set forth in this Agreement, and in complete and full satisfaction of the Note, on the date of the closing of the Primary Transactions (“Closing Date”), the Bank shall pay to SunTrust cash in an amount equal to 35% of the outstanding principal amount of the Note as of the Closing Date, plus 100% of the unpaid and accrued interest thereon, accruing, from and after May 11, 2011, and

continuing to accrue through and including the Closing Date, less and except any such interest as theretofore paid, at the rate of 8% per annum (“Debt Settlement”).

SunTrust shall mark the original Note cancelled upon receiving the payment made under this Section 1, and promptly deliver such original Note marked cancelled to the Bank.

The Subordinated Debt Loan Agreement between the Bank and SunTrust, dated June 30, 2008 (the “Loan Agreement”), and the other Loan Documents (as such term is defined in the Loan Agreement) shall be terminated immediately upon SunTrust’s receipt of the payment.

2. Subject to the conditions set forth in this Agreement, the Bank also agrees to purchase from SunTrust and SunTrust agrees to sell to the Bank, on the Closing Date, the Preferred Stock for cash in an amount equal to 25% of the aggregate liquidation preference thereof, plus 100% of the unpaid and accrued dividends, accruing at the rate of 8% per annum: (i) as to $7,500,000 of the Preferred Stock from and after December 30, 2010, and continuing to accrue thereon through and including the Closing Date; and (ii) as to $5,000,000 of the Preferred Stock from and after February 28, 2011, and continuing to accrue thereon through and including the Closing Date (“Preferred Stock Repurchase”).

Upon receipt of the payment made under this Section 2, SunTrust shall promptly deliver to the Bank the two stock certificates representing the Preferred Stock.

3. Subject to the receipt of appropriate regulatory approval, the Bank agrees to promptly pay or promptly reimburse all reasonable and fully itemized costs and expenses of SunTrust in connection with the negotiation, execution and delivery of this Agreement as of the Closing Date, including the reasonable and fully itemized fees and disbursements of counsel for SunTrust.

4. The closing of the Debt Settlement and Preferred Stock Repurchase is conditioned upon the following:

(a) all the other conditions precedent to the closing of the Primary Transactions shall have been satisfied or waived;

(b) the Office of the Comptroller of the Currency shall have approved the Debt Settlement and Preferred Stock Repurchase;

(c) the Federal Reserve Bank of Richmond shall have approved FNB’s payment of interest on certain junior subordinated debt to the extent that such payment is a condition to FNB’s providing shareholder consent for the Preferred Stock Repurchase; and

(d) the respective representations and warranties of the Bank and SunTrust shall be true and correct in all material respects at and as of the Closing Date.

2

5. If the closing of the Debt Settlement and Preferred Stock Repurchase has not occurred by October 15, 2011, either SunTrust or the Bank may terminate this Agreement upon giving the other party written notice at any time after that date.

6. The Bank represents and warrants to SunTrust as follows as of the date hereof: (a) the Bank has the requisite corporate power and authority to enter into this Agreement and any agreement or instrument to be executed and delivered in connection with or pursuant hereto, and, subject to receipt of required regulatory approval and shareholder approval, to perform its obligations hereunder and to consummate the transactions contemplated hereby; (b) the Bank has taken all requisite corporate action and obtained due authorization (other than receipt of required shareholder approval) for its execution, delivery and performance of this Agreement and its consummation of the transactions contemplated hereby; (c) the Bank has duly executed and delivered this Agreement, and assuming due authorization, execution and delivery of this Agreement by SunTrust, this Agreement constitutes a legal, valid and binding agreement enforceable against the Bank in accordance with its terms, subject to bankruptcy, insolvency, reorganization, moratorium and similar laws of general application relating to or affecting creditors’ rights and to general equity principles; (d) the execution and delivery of this Agreement, the consummation of the transactions contemplated hereby, and the compliance with any of the provisions hereof by the Bank do not (i) violate or conflict with any provisions of its organizational documents, (ii) result in a default (or an event that, with notice or lapse of time or both, would become a default), or give rise to any right of termination or buy-out by any third party, cancellation, amendment or acceleration of any obligation, or the loss of any benefit under any contract to which it or any of its subsidiaries is a party or by which it or any of its subsidiaries or any of their respective assets or properties is bound or affected, (iii) result in the creation of a lien on any of its issued and outstanding equity interests or on any of its assets or the assets of any of its subsidiaries, or (iv) violate or conflicts with any law applicable to it or any of its subsidiaries, or any of the properties, businesses or assets of any of the foregoing, other than such exceptions in the case of each of clauses (ii) and (iii) above as would not, individually or in the aggregate, reasonably be expected to materially impair or delay the Bank’s ability to perform each of its obligations hereunder or to consummate the transactions contemplated hereby; and (e) to the extent applicable, the Bank has obtained or will obtain prior to the Closing Date any consent, approval, authorization or permit, made any registration, declaration or filing with, and submitted any notification to, any regulatory authority or any other person required to execute and deliver this Agreement, consummate the transactions contemplated hereby, or comply with any of the provisions hereof.

7. SunTrust represents and warrants to the Bank as follows as of the date hereof: (a) SunTrust has the requisite corporate power and authority to enter into this Agreement and any agreement or instrument to be executed and delivered in connection with or pursuant hereto, to perform its obligations hereunder, and to consummate the transactions contemplated hereby; (b) SunTrust has taken all requisite corporate action

3

and obtained due authorization for its execution, delivery and performance of this Agreement and its consummation of the transactions contemplated hereby; (c) SunTrust has duly executed and delivered this Agreement, and assuming due authorization, execution and delivery of this Agreement by the Bank, this Agreement constitutes a legal, valid and binding agreement enforceable against SunTrust in accordance with its terms, subject to bankruptcy, insolvency, reorganization, moratorium and similar laws of general application relating to or affecting creditors’ rights and to general equity principles; (d) the execution and delivery of this Agreement, the consummation of the transactions contemplated hereby, and the compliance with any of the provisions hereof by SunTrust do not (i) violate or conflict with any provisions of its organizational documents, (ii) result in a default (or an event that, with notice or lapse of time or both, would become a default), or give rise to any right of termination or buy-out by any third party, cancellation, amendment or acceleration of any obligation, or the loss of any benefit under any contract to which it or any of its subsidiaries is a party or by which it or any of its subsidiaries or any of their respective assets or properties is bound or affected, (iii) result in the creation of a lien on any of its issued and outstanding equity interests or on any of its assets or the assets of any of its subsidiaries, or (iv) violate or conflicts with any law applicable to it or any of its subsidiaries, or any of the properties, businesses or assets of any of the foregoing, other than such exceptions in the case of each of clauses (ii) and (iii) above as would not, individually or in the aggregate, reasonably be expected to materially impair or delay SunTrust’s ability to perform each of its obligations hereunder or to consummate the transactions contemplated hereby; and (e) to the extent applicable, SunTrust has obtained or will obtain prior to the Closing Date any consent, approval, authorization or permit, made any registration, declaration or filing with, and submitted any notification to, any regulatory authority or any other person required to execute and deliver this Agreement, consummate the transactions contemplated hereby, or comply with any of the provisions hereof.

8. TO INDUCE SUNTRUST TO AGREE TO THE TERMS OF THIS AGREEMENT, THE BANK (BY ITS EXECUTION BELOW) REPRESENTS AND WARRANTS THAT AS OF THE DATE OF THIS AGREEMENT THERE ARE NO CLAIMS OR OFFSETS AGAINST OR DEFENSES OR COUNTERCLAIMS TO THE BANK’S OBLIGATIONS UNDER THE LOAN AGREEMENT AND THE OTHER LOAN DOCUMENTS, AND WAIVES ANY AND ALL SUCH CLAIMS, OFFSETS, DEFENSES, OR COUNTERCLAIMS, WHETHER KNOWN OR UNKNOWN, ARISING PRIOR TO THE DATE OF THIS AGREEMENT AND RELEASES AND DISCHARGES SUNTRUST AND ITS OFFICERS, DIRECTORS, EMPLOYEES, AGENTS, SHAREHOLDERS, AFFILIATES, AND ATTORNEYS (COLLECTIVELY THE “RELEASED PARTIES”) FROM ANY AND ALL OBLIGATIONS, INDEBTEDNESS, LIABILITIES, CLAIMS, RIGHTS, CAUSES OF ACTION, OR DEMANDS WHATSOEVER, WHETHER KNOWN OR UNKNOWN, SUSPECTED OR UNSUSPECTED, AT LAW OR IN EQUITY, WHICH THE BANK NOW HAS OR MAY HAVE AGAINST ANY RELEASED PARTY ARISING PRIOR TO THE DATE HEREOF AND FROM OR IN CONNECTION WITH THE LOAN AGREEMENT, THE OTHER LOAN DOCUMENTS, OR THE TRANSACTIONS CONTEMPLATED THEREBY.

4

9. This Agreement shall be governed by, and construed in accordance with, the laws of the State of Georgia, without regard to conflict of laws provisions of the State of Georgia or of any other state.

10. This Agreement may be executed in more than one counterparts, each of which will be deemed to be an original but all of which together will constitute one and the same instrument.

11. This Agreement constitutes the entire agreement between the parties hereto with respect to the subject matter hereof and supersedes all other prior agreements or understandings, both written and oral, between the parties or any of them with respect to the subject matter hereof. The provisions and representations and warranties set forth in Sections 6, 7, 8, and 9 and this Section 11 shall survive any termination of this Agreement. The only representations and warranties made by the parties hereto with respect to the subject matter hereof are the representations and warranties contained in or made pursuant to this Agreement.

IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be executed on their behalf as of the date first above written.

5