Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement on Form S-1 filed pursuant to Rule 462(b) of the Securities Act of 1933 of the reference to our firm under the caption “Experts” and to the use of our report dated March 31, 2015, with respect to the financial statements of Immune Design Corp. included in Amendment No. 2 to the Registration Statement (Form S-1 No. 333-203241) and related Prospectus of Immune Design Corp. for the registration of its common stock.

/s/ Ernst & Young LLP

Seattle, Washington

April 15, 2015