Exhibit 10.5

AMENDED AND RESTATED CASUAL MALE RETAIL GROUP, INC.

LONG-TERM INCENTIVE PLAN

Section 1. Establishment and Purpose

Casual Male Retail Group, Inc. (the “Company”) established a long-term incentive plan named the 2008 Casual Male Retail Group, Inc. Long-Term Incentive Plan, for the purpose of supporting the Company’s ongoing efforts to attract, retain and develop exceptional talent and enable the Company to provide incentives directly linked to the Company’s short and long-term objectives and increases in shareholder value. This document amends and restates the 2008 Casual Male Retail Group, Inc. Long-Term Incentive Plan in its entirety (as amended and restated, the “Plan”), effective as of May 3, 2010.

Section 2. Definitions

When used herein, the following capitalized terms shall have the meanings assigned to them, unless the context clearly indicates otherwise. Capitalized terms used herein and not defined shall have the meanings assigned to them in the Company’s 2006 Incentive Compensation Plan (the “Incentive Compensation Plan”).

Section 3. Establishment of Fiscal Year Target and Awards

Within 90 days after the start of each fiscal year of the Company, the Compensation Committee of the Board of Directors (the “Committee”) will establish specific goals, in accordance with Section 8(b) of the Incentive Compensation Plan, based upon the achievement of one or more of the performance criteria set forth therein, upon which each Participant’s Award shall be determined for that fiscal year. The Committee may establish a threshold goal (which if not achieved will result in no Award being payable), target and maximum goals for each Participant. The goals, performance criteria, and targets used may vary from one Participant to another in the sole discretion of the Committee. The Awards are intended to qualify as “performance-based compensation” exempt from the deduction limitations imposed under Section 162(m) of the Code. Accordingly the Awards shall be subject to all of the requirements in Section 8 of the Incentive Compensation Plan and such other terms and conditions as shall be necessary for the Awards to qualify for the exception to Section 162(m) of the Code.

To be eligible for an Award, a Participant must be actively employed as of the last day of the fiscal year and at the time the Award is distributed. Notwithstanding the foregoing, if a Participant retires on or after age 65 or leaves employment due to death or Disability (as defined in the Incentive Compensation Plan), before the end of the Plan year, he/she will receive a pro-rated Award calculated as set forth below. In order to be eligible to receive an Award under this Plan, a Participant must comply with all applicable state and federal regulations and Company policies.

If a Participant became a Participant during a fiscal year, but after the start of the fiscal year but is otherwise eligible, the Award to such Participant shall be calculated using a pro rata portion of such Participant’s Target Cash Value, such pro rata portion to be determined by dividing the total number of calendar days elapsed during such fiscal year in which such Participant was actually employed by the Company as a Participant by the total number of calendar days in such fiscal year.

Section 4. Validity; Irrevocable Election

A Participant shall elect to receive his or her Award for a given fiscal year by filing an Irrevocable Election Agreement, substantially in the form of Exhibit A attached hereto with the Company by no later than six (6) months before the close of the fiscal year in which the Award is to be earned. To be effective, a fully executed Irrevocable Election Agreement must be submitted by the Participant to the Company’s Human Resources Department, with a copy to its General Counsel, prior to the deadline and shall thereafter be irrevocable and non-modifiable. A Participant who does not, for any reason, file an Irrevocable Election Agreement with the Company on a timely basis shall be deemed to have elected to have his or her entire Award paid in Cash. Each Award election shall remain in full force and effect for the three installment periods referred to below in Section 5.

Section 5. Election; Award Determination; and Distribution

A Participant shall select in his or her Irrevocable Election Agreement for a fiscal year the portion of any Award for such fiscal year that will be payable in Cash, Restricted Stock or Options. The Committee may, in its discretion, require that all or any portion of a Participant’s Award be paid in Restricted Stock and/or Options and not in cash. For the purpose of dividing an Award into the applicable Award Payment Choice, Options shall be valued at their Black-Scholes Value on the Grant Date of the Award and each Share of Restricted Stock shall be valued at its Fair Market Value on the Grant Date of the Award.

Awards will vest in three equal increments on the first, second and third anniversaries of the Grant Date of the Award, provided that the Participant’s Continuous Service continues through the applicable vesting date. For example, if the Committee determines that Participants are entitled to make elections for the 2010 fiscal year, and grants the Awards on April 1, 2011 the Awards will vest in three equal installments on April 1, 2012, April 1, 2013 and April 1, 2014, respectively.

Except in the event of a Change in Control, all unpaid Cash, unvested Restricted Stock and unvested Options to which a Participant would otherwise be entitled shall be forfeited immediately upon the Participant’s Separation from Service.

Awards made hereunder are being made pursuant to the Incentive Compensation Plan. In the event that, at the time an Award is granted, the Company does not have a sufficient number of shares remaining unissued under the Incentive Compensation Plan to issue such Award in Restricted Stock and/or Options, then, regardless of the election made by a Participant, such Award shall be paid in Cash to the extent of such insufficiency.

Section 6. Change in Control.

If and to the extent that it would not violate the requirements of Section 409A of the Code, in the event of a Change in Control prior to a Participant’s Separation from Service, the full value of the Participant’s Award (including any remaining cash installments (and Interest thereon) that otherwise would have been payable to the Participant), shall immediately vest and be distributed as a lump sum to the Participant, as soon as practicable (but in no event more than five (5) business days) following the Change in Control.

Section 7. No Acceleration of Benefits

In no event shall the acceleration of the time or schedule of any payment under the Plan be permitted, except to the extent that such acceleration would not violate Section 409A of the Code and the Treasury Regulations and other applicable guidance issued thereunder.

Section 8. Amendment and Termination

This Plan may be amended or terminated in any respect at any time by the Committee; provided, however, that no amendment or termination of the Plan shall be effective to reduce any benefits that accrue and are vested before the adoption of such amendment or termination. If and to the extent permitted without violating the requirements of Section 409A of the Code, the Committee may require that the Awards of all Participants (including, without limitation, any remaining benefits payable to Participants receiving distributions in installments at the time of the termination) be distributed as soon as practicable after such termination, notwithstanding any elections by Participants with regard to the timing or form in which their benefits are to be paid. If and to the extent that the Committee does not accelerate the timing of distributions on account of

the termination of the Plan pursuant to the preceding sentence, payment of any remaining benefits under the Plan shall be made at the same times and in the same manner as such distributions would have been made under the terms of the Plan, as in effect at the time the Plan is terminated.

Section 9. Unfunded Obligation

The obligations of the Company to pay any benefits under the Plan shall be unfunded and unsecured, and any payments under the Plan shall be made from the general assets of the Company. Participants’ rights under the Plan are not assignable or transferable except to the extent that such assignment or transfer is permitted under the terms of the Incentive Compensation Plan.

Section 10. Withholding

The Participants and personal representatives shall bear any and all federal, state, local or other taxes imposed on benefits under the Plan. The Company may deduct from any distributions under the Plan the amount of any taxes required to be withheld from such distribution by any federal, state or local government, and may deduct from any compensation or other amounts payable to the Participant the amount of any taxes required to be withheld with respect to any other amounts under the Plan by any federal, state or local government.

Section 11 Applicable Law

This Plan shall be construed and enforced in accordance with the laws of the State of Delaware, except to the extent superseded by federal law.

Section 12. Administration and Interpretation

The Plan will be administered by the Committee. The Committee will have broad authority to determine target Metrics, select performance objectives, adopt rules and regulations relating to the Plan and make decisions and interpretations regarding the provisions of the Plan, all subject to the terms and conditions of the Incentive Compensation Plan, including without limitation those set forth in Section 8 of the Incentive Compensation Plan. Benefits due and owing to a Participant under the Plan shall be paid when due without any requirement that a claim for benefits be filed. However, any Participant who has not received the benefits to which Participant believes himself or herself entitled may file a written claim with the Committee, who shall act on the claim within thirty days. Any decisions or interpretations by the Committee relating to benefits under the Plan shall be binding and conclusive on all affected parties. If a Participant’s employment agreement conflicts with any provision of this Plan, the language of the employment agreement shall govern.

Section 13. Code Section 409A

It is intended that the Awards granted pursuant to this Plan be exempt from Section 409A of the Code (“Section 409A”) because it is believed (i) the Awards payable in cash should qualify for the short-term deferral exception contained in Treasury Regulation §1.409A-1(a)(4), (ii) any Options granted pursuant to the Plan will have an exercise price that may never be less than the Fair Market Value of a Share on the Grant Date and the other requirements for the exemption of such options

under Treasury Regulation §1.409A-1(a)(5)(i)(A) should be met; and (iii) any Shares of Restricted Stock granted under the Plan should be exempt as an award of restricted property pursuant to Treasury Regulation §1.409A-1(a)(6). The provisions of the Plan shall be interpreted in a manner consistent with that intent.

The Committee, in its sole discretion, and without the consent of any Participant or Beneficiary, may amend the provisions of this Plan to the extent that the Committee determines that such amendment is necessary or appropriate in order for the Awards made pursuant to the Plan to be exempt from the requirements of Section 409A, or if and to the extent that the Committee determines that Awards are not so exempt, to amend the Plan (and any agreements relating to any Awards) in such manner as the Committee shall deem necessary or appropriate to comply with the requirements of Section 409A.

Notwithstanding the foregoing, the Company does not make any representation to any Participant or Beneficiary that the Awards made pursuant to this Plan are exempt from, or satisfy, the requirements of Section 409A, and the Company shall have no liability or other obligation to indemnify or hold harmless any Participant or Beneficiary for any tax, additional tax, interest or penalties that the Participant or Beneficiary may incur in the event that any provision of the Plan or any Award agreement, or any amendment or any modification thereof, or any other action taken with respect thereto, is deemed to violate any of the requirements of Section 409A.

Section 14. No Assignment.

Neither any Participant nor any Beneficiary nor any other person shall have any right to assign the rights to receive any payments or benefits hereunder, in whole or in part, which payments and benefits are non-assignable and non-transferable, whether voluntarily, or involuntarily.

THIS SPACE IS LEFT BLANK INTENTIONALLY

EXHIBIT A

AMENDED AND RESTATED CASUAL MALE RETAIL GROUP, INC.

LONG-TERM INCENTIVE PLAN AWARD

IRREVOCABLE ELECTION AGREEMENT

I, ______________________, hereby elect to receive my Award (as defined in the Amended and Restated Casual Male Retail Group, Inc. Long-Term Incentive Plan (the “Plan”)) based upon the performance for the Company’s fiscal year ending ______ as follows:

[NOTE: Payments will vest in three (3) equal installments. You have the opportunity to decide the Award Payment Choice(s): cash, CMRG restricted stock or CMRG stock options for each equal installment. Your selected option(s) for any given year must equal 100%.]

I understand and acknowledge that this election is irrevocable and will remain in effect up through the third and final installment.

I understand and acknowledge that I must be employed on the dates each portion of the Award vests in order to qualify for payment of that installment.

I understand and acknowledge that if there is any conflict between this form or any part of it and the Plan, the provisions of the Plan shall govern.

I have hereunto set my hand and seal this __ day of _________, 20__.