Exhibit 5.1

Laura A. Berezin

+1 650 843 5128

lberezin@cooley.com

September 16, 2016

Immune Design Corp.

1616 Eastlake Ave. E., Suite 310

Seattle, Washington 98102

Ladies and Gentlemen:

We have acted as counsel to Immune Design Corp., a Delaware corporation (the “Company”), in connection with the offering of up to 5,520,000 shares of the Company’s common stock, par value $0.001 (the “Shares”), including up to 720,000 Shares that may be sold pursuant to the exercise of an option to purchase additional Shares, pursuant to a Registration Statement on Form S-3 (Registration Statement No. 333-206324) (the “Registration Statement”), filed with the Securities and Exchange Commission (the “Commission”) under the Securities Act of 1933, as amended (the “Act”) and the prospectus dated December 29, 2015 included within the Registration Statement (the “Base Prospectus”), and the prospectus supplement relating to the Shares dated September 15, 2016 filed with the Commission pursuant to Rule 424(b) of the Rules and Regulations of the Act (the “Prospectus Supplement”). The Base Prospectus and the Prospectus Supplement are collectively referred to as the “Prospectus.” The Shares are to be issued and sold by the Company as described in the Registration Statement and the Prospectus.

In connection with this opinion, we have examined and relied upon the Registration Statement, the Prospectus, the Company’s certificate of incorporation and bylaws, as currently in effect, and the originals or copies certified to our satisfaction of such records, documents, certificates, memoranda and other instruments as in our judgment are necessary or appropriate to enable us to render the opinion expressed below. As to certain factual matters, we have relied upon a certificate of an officer of the Company and have not sought to independently verify such matters.

In rendering this opinion, we have assumed the genuineness and authenticity of all signatures on original documents, the genuineness and authenticity of all documents submitted to us as originals, the conformity to originals of all documents submitted to us as copies; and the accuracy, completeness and authenticity of certificates of public officials.

Our opinion herein is expressed solely with respect to the federal laws of the United States and the General Corporation Law of the State of Delaware. Our opinion is based on these laws as in effect on the date hereof, and we disclaim any obligation to advise you of facts, circumstances, events or developments which hereafter may be brought to our attention and which may alter, affect or modify the opinion expressed herein. We express no opinion as to whether the laws of any particular jurisdiction other than those identified above are applicable to

COOLEY LLP    3175 HANOVER STREET    PALO ALTO, CA    94304-1130

T: (650) 843-5000  F: (650) 849-7400  COOLEY.COM

Immune Design Corp.

September 16, 2016

Page Two

the subject matter hereof. We are not rendering any opinion as to compliance with any federal or state antifraud law, rule or regulation relating to securities, or to the sale or issuance thereof.

On the basis of the foregoing, and in reliance thereon, we are of the opinion that the Shares, when sold in accordance with the Registration Statement and the Prospectus, will be validly issued, fully paid and nonassessable.

We consent to the reference to our firm under the caption “Legal Matters” in the Prospectus, the filing of this opinion as an exhibit to a Current Report on Form 8-K and the incorporation by reference of this opinion in the Registration Statement.

Sincerely,

COOLEY LLP    3175 HANOVER STREET    PALO ALTO, CA    94304-1130

T: (650) 843-5000  F: (650) 849-7400  COOLEY.COM