Exhibit 10.4

FAMILY DOLLAR STORES, INC.

FAMILY DOLLAR, INC.

FIRST AMENDMENT

Dated as of November 17, 2010

to

NOTE PURCHASE AGREEMENT

Dated as of September 27, 2005

and to the

NOTES described below

$169,000,000 5.41% Series 2005-A Senior Notes, Tranche A, due September 27, 2015

$81,000,000 5.24% Series 2005-A Senior Notes, Tranche B, due September 27, 2015

FIRST AMENDMENT TO NOTE PURCHASE AGREEMENT AND NOTES

This First Amendment dated as of November 17, 2010 (the or this “First Amendment”) to the Note Purchase Agreement dated as of September 27, 2005 and to the Notes (as such term is defined below) is between Family Dollar Stores, Inc., a Delaware corporation (“FDSI”), Family Dollar, Inc. (“FDI”, and, together with FDSI, the “Obligors”), and each of the institutions which is a signatory to this First Amendment (collectively, the “Noteholders”).

R E C I T A L S:

A. The Obligors and each of the Noteholders have heretofore entered into the Note Purchase Agreement dated as of September 27, 2005 (the “Note Agreement”). The Obligors have heretofore issued the $169,000,000 5.41% Series 2005-A Senior Notes, Tranche A, due September 27, 2015 and 5.24% Series 2005-A Senior Notes, Tranche B due September 27, 2015 (the “Notes”) dated September 27, 2005 pursuant to the Note Agreement.

B. The Obligors and the Noteholders now desire to amend the Note Agreement and the Notes in the respects, but only in the respects, hereinafter set forth.

C. Capitalized terms used herein shall have the respective meanings ascribed thereto in the Note Agreement unless herein defined or the context shall otherwise require.

D. All requirements of law have been fully complied with and all other acts and things necessary to make this First Amendment a valid, legal and binding instrument according to its terms for the purposes herein expressed have been done or performed.

NOW, THEREFORE, upon the full and complete satisfaction of the conditions precedent to the effectiveness of this First Amendment set forth in Section 4.1 hereof, and in consideration of good and valuable consideration the receipt and sufficiency of which is hereby acknowledged, the Obligors and the Noteholders do hereby agree as follows:

SECTION 1. AMENDMENTS.

Section 1.1. The Note Agreement and Notes shall be and are hereby amended by deleting the words “Obligor”, “Obligors”, “an Obligor”, “any Obligor” and “FDI” from the Note Agreement and the Notes and replacing them with the word “FDSI”.

Section 1.2. Section 1.2 of the Note Agreement shall be and hereby is amended by the adding the following new sentences at the end of such Section 1.2:

“If on any date a rating issued by S&P or Moody’s, as the case may be, evidenced by either a confidential rating letter, and addressed to FDSI, or the public credit rating of FDSI’s long-term unsecured debt (each such rating, a “Rating”) is less than (a) BBB- or its equivalent but equal to or greater than BB- or its equivalent as rated

-1-

by S&P, or (b) Baa3 or its equivalent but equal to greater than Ba3 or its equivalent as rated by Moody’s, then from and including such date interest on the Notes shall accrue at a per annum interest rate equal to 1.00% per annum above the interest rate otherwise applicable with respect to such Notes. If (a) on any date a Rating is less than (i) BB- or its equivalent as rated by S&P, or (ii) Ba3 or its equivalent as rated by Moody’s, or (b) on any date following ninety (90) days after the date hereof, no Rating has been assigned, then from and including such date interest on the Notes shall accrue at a per annum interest rate equal to 2.00% per annum above the interest rate otherwise applicable with respect to such Notes. FDSI shall ensure that (a) the applicable rating agency continuously monitors and evaluates the Rating on a maintenance basis to determine whether the Rating should be upgraded, downgraded or remain static, and (b) that such rating agency provides FDSI with prompt notice of any upgrade or downgrade in the Rating. FDSI shall promptly, but in any event within five (5) Business Days following its receipt of a confidential rating letter, public credit rating or similar document relating to the Rating required be maintained hereunder or any change in such Rating, forward a copy of such confidential rating letter, public credit rating or similar document to each holder of Notes in accordance with the terms of Section 18.”

Section 1.3. Section 10.1 of the Note Agreement shall be and is hereby amended by deleting “60%” as used therein and replacing it with “50%”.

Section 1.4. Section 10.3 of the Note Agreement shall be and is hereby amended by deleting “20%” as used therein and replacing it with “10%”.

Section 1.5. Section 10.4(j) of the Note Agreement shall be and is hereby amended in its entirety to read as follows:

“(j) in addition to the Liens described above, any other Liens securing Debt or other obligations not permitted above, including Liens securing Priority Debt of FDSI or any Restricted Subsidiary, provided that such Priority Debt does not exceed the limitations set forth in Section 10.3, and, provided further that, no such Liens under this Section 10.4(j) may secure any obligations under any Parity Debt Agreement unless the obligations under this Agreement and the Notes are equally and ratably secured pursuant to an agreement reasonably satisfactory to the Required Holders.”

Section 1.6. Section 10.7 of the Note Agreement shall be and is hereby amended by deleting “75%” as used therein and replacing it in each place where it occurs in such section with “90%”.

-2-

Section 1.7. The following new Section 10.11 shall be inserted immediately following Section 10.10 of the Note Agreement:

Section 10.11. Additional Restrictions. In addition to and not in limitation of any of the restrictions to which FDSI or any of its Subsidiaries is subject pursuant to this Agreement, FDSI agrees that if FDSI or any of its Subsidiaries is or becomes subject to any covenant, agreement or similar restriction, including without limitation those which may be expressed as “events of default” (together with the relevant definitions, an “Additional Covenant”), for the benefit of any bank, institutional lender, or any other holder thereof, whether in an existing Debt agreement, through an amendment to an existing Debt agreement (including through an amendment of the definitions used in a covenant) or through FDSI or any of its Subsidiaries entering into a new Debt agreement (each, an “Other Debt Agreement,” which in all cases shall include all Parity Debt Agreements), which Additional Covenant is in addition to, or more restrictive than, the covenants, agreements and restrictions to which FDSI and its Subsidiaries are subject pursuant to this Agreement, then promptly upon the effectiveness of such Additional Covenant this Agreement shall be deemed to have been amended and such Additional Covenant (together with all relevant definitions) will be deemed to be incorporated herein. Notwithstanding the foregoing, provided that no Default or Event of Default is then in existence, any amendment, waiver or elimination of any Additional Covenant in accordance with the terms of the relevant Other Debt Agreement, or the termination of such Other Debt Agreement, shall, at the written request of FDSI, constitute an amendment, waiver, elimination or termination, as the case may be, of such Additional Covenant incorporated herein pursuant to this Section 10.11; provided, however, if FDSI has requested such amendment, waiver, elimination or termination, as the case may be, then if any fee or other form of consideration (other than the payment of contracted principal, interest, break-funding payments and prepayment premiums or similar charges) is given to any bank or institutional lender under the Other Debt Agreement for the purpose of such amendment, waiver, elimination or termination, as the case may be, the holders of the Notes shall receive equivalent consideration (on a pro rata basis), substantially concurrently with such bank or institutional lender. Upon the written request of FDSI or the Required Holders, FDSI and the holders of the Notes shall enter into a written agreement memorializing and acknowledging such incorporation of such Additional Covenant (and related definitions) or the amendment, waiver, elimination or termination thereof, as the case may be.

-3-

FDSI shall provide to the holders of Notes a certified copy of the Other Debt Agreement containing the addition of such Additional Covenant thereto, or the amendment, waiver, elimination or termination thereof, as the case may be.

Section 1.8. Section 22.3 of the Note Agreement shall be and is hereby amended by adding the following sentence at the end of Section 22.3:

“For purposes of determining compliance with the covenants set out in this Agreement, any election by FDSI to measure an item of indebtedness using fair value (as permitted by Statement of Financial Accounting Standards Nos. 157 or 159) shall be disregarded and such determination shall be made by valuing indebtedness at 100% of the outstanding principal thereof.”

Section 1.9. Schedule B of the Note Agreement shall be and is hereby amended by:

(a) deleting subclause (z) of the definition of “Priority Debt”, deleting the word “and” immediately following subclause (y) in the definition of “Priority Debt”, and by amending subclause (y) of the definition of “Priority Debt” in its entirety to read as follows:

“(y) all Debt of any Restricted Subsidiary which has guaranteed, or is otherwise obligated as a co-borrower or co-obligor under, the Notes,”;

(b) adding the following definition of the term “Parity Debt Agreement” in alphabetical order to such Schedule B:

“Parity Debt Agreement” means (a) the Bank Credit Agreement, and (b) any other agreement or instrument, entered into individually or in concert or in connection with any other agreement or instrument, creating, evidencing or having borrowing capacity of Debt outstanding of FDSI and/or any Subsidiary equal to or greater than $50,000,000 in the aggregate.

; and

(c) amending the definition of the term “Bank Credit Agreement” as follows:

“Bank Credit Agreement” means (i) the Credit Agreement dated as of August 24, 2006 by and among FDSI, certain Subsidiaries of FDSI named therein, Wells Fargo Bank, National Association (as successor by merger to Wachovia Bank, National Association), as Administrative Agent, Swingline Lender and Fronting Bank, and various other lenders named therein and (ii) the

-4-

Credit Agreement dated as of November 17, 2010 by and among FDSI, Wells Fargo Bank, National Association, as administrative agent, and various other lenders named therein, each as amended, restated, joined, supplemented or otherwise modified from time to time, and any renewals, extensions or replacements thereof, which constitute the primary bank credit facilities of FDSI and its Subsidiaries.

Section 1.10. Each Note shall be amended by adding the following sentence to the end of the first paragraph of such Note:

“The per annum interest rate applicable on this Note is subject to adjustment in accordance with Section 1.2 of the Note Purchase Agreement.”

SECTION 2. RELEASE OF FDI.

From and after the effectiveness of this First Amendment, (a) FDI shall be released as a co-obligor under the Notes and the Note Agreement and (b) all Subsidiary Guarantors shall be released under the Notes and the Note Agreement; provided that the Obligor has satisfied the conditions set forth in Section 2.3(b) of the Note Agreement.

SECTION 3. REPRESENTATIONS AND WARRANTIES OF THE OBLIGORS.

Section 3.1. To induce the Noteholders to execute and deliver this First Amendment (which representations shall survive the execution and delivery of this First Amendment), each Obligor represents and warrants to the Noteholders that:

(a) this First Amendment has been duly authorized, executed and delivered by it and this First Amendment constitutes the legal, valid and binding obligation, contract and agreement of the Obligors enforceable against it in accordance with its terms, except as enforcement may be limited by bankruptcy, insolvency, reorganization, moratorium or similar laws or equitable principles relating to or limiting creditors’ rights generally;

(b) the Note Agreement, as amended by this First Amendment, constitutes the legal, valid and binding obligations, contracts and agreement of FDSI enforceable against it in accordance with its terms, except as enforcement may be limited by bankruptcy, insolvency, reorganization, moratorium or similar laws or equitable principles relating to or limiting creditors’ rights generally;

(c) the execution, delivery and performance by it of this First Amendment (i) has been duly authorized by all requisite corporate action and, if required, shareholder action, (ii) does not require the consent or approval of any governmental or regulatory body or agency, and (iii) will not (A) violate (1) any provision of law, statute, rule or regulation or its certificate of incorporation or bylaws, (2) any order of any court or any

-5-

rule, regulation or order of any other agency or government binding upon it, or (3) any provision of any material indenture, agreement or other instrument to which it is a party or by which its properties or assets are or may be bound, including, without limitation, the Bank Credit Agreement or (B) result in a breach or constitute (alone or with due notice or lapse of time or both) a default under any indenture, agreement or other instrument;

(d) as of the date hereof and after giving effect to this First Amendment, no Default or Event of Default has occurred which is continuing; and

(e) it has received and is in possession of an indicative rating of its long-term unsecured debt of BBB- or greater from S&P and Baa3 or greater from Moody’s, and such letters from each of Moody’s and S&P confirming such ratings remain in full force and effect.

SECTION 4. CONDITIONS TO EFFECTIVENESS OF THIS FIRST AMENDMENT.

Section 4.1. This First Amendment shall not become effective until, and shall become effective when, each and every one of the following conditions shall have been satisfied:

(a) executed counterparts of this First Amendment, duly executed by the Obligors and the holders of 100% of the outstanding principal of the Notes, shall have been delivered to the Noteholders;

(c) the representations and warranties of each Obligor set forth in Section 3 hereof are true and correct on and with respect to the date hereof;

(d) FDSI shall have paid the reasonable fees and expenses of Chapman and Cutler LLP, counsel to the Noteholders, in connection with the negotiation, preparation, approval, execution and delivery of this First Amendment;

(e) FDI shall have been released from its obligations as a co-obligor under the Bank Credit Agreement;

(f) If any Subsidiary, including without limitation FDI, is at such time a party to the Bank Credit Agreement as a guarantor, borrower, or otherwise as an obligor, such Subsidiary shall have become a Subsidiary Guarantor by satisfying all of the conditions contained in Section 9.8 of the Note Agreement; and

(g) each of the Noteholders shall have received a non-refundable amendment fee equal to 0.25% of the outstanding principal amount of the Notes (the “Amendment Fee”) held by such holder, with such Amendment Fee to be paid by the Obligors to each Noteholder’s account specified in Schedule A to the Note Purchase Agreement (or otherwise as shall have specified to the Obligors in writing prior to the date hereof).

-6-

Upon satisfaction of all of the foregoing conditions, this First Amendment shall become automatically effective.

SECTION 5. MISCELLANEOUS.

Section 5.1. This First Amendment shall be construed in connection with and as part of each of the Notes and the Note Agreement, and except as modified and expressly amended by this First Amendment, all terms, conditions and covenants contained in the Note Agreement and the Notes are hereby ratified and shall be and remain in full force and effect.

Section 5.2. Any and all notices, requests, certificates and other instruments executed and delivered after the execution and delivery of this First Amendment may refer to the Note Agreement without making specific reference to this First Amendment but nevertheless all such references shall include this First Amendment unless the context otherwise requires.

Section 5.3. The descriptive headings of the various Sections or parts of this First Amendment are for convenience only and shall not affect the meaning or construction of any of the provisions hereof.

Section 5.4. This First Amendment shall be governed by and construed in accordance with New York law.

Section 5.5. The execution hereof by you shall constitute a contract between us for the uses and purposes hereinabove set forth, and this First Amendment may be executed in any number of counterparts, each executed counterpart constituting an original, but all together only one agreement.

[Signature pages immediately follow.]

-7-

IN WITNESS WHEREOF, the parties hereto have executed and delivered this First Amendment as of the date first written above.

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

-2-

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page

Family Dollar Stores, Inc.

First Amendment to Note Purchase Agreement and Notes

Signature Page