Exhibit 5.1

December 15, 2015

Board of Directors

Bank of the James Financial Group, Inc.

828 Main Street

Lynchburg, VA 24504

Registration Statement on Form S-1

Ladies and Gentlemen:

We are acting as counsel to Bank of the James Financial Group, Inc., a Virginia corporation (the “Company”), in connection with the proposed registration by the Company of 1,000,000 shares of its common stock, par value $2.14 per share (the “Common Stock”) and the related preparation and filing by the Company with the Securities and Exchange Commission (“Commission”) of a Registration Statement on Form S-1 (the “Registration Statement”) that is being filed herewith under the Act (such Registration Statement, as amended or supplemented, is hereinafter referred to as the Registration Statement). These Shares were previously issued by the Company in a private placement transaction pursuant to Rule 506 of Regulation D and/or Section 4(a)(2) of the Securities Act of 1933, as amended (the “Act”) (the “Private Placement”). The shares of Common Stock to be registered pursuant to the Registration Statement (the “Registrable Shares”) are being registered in accordance with the requirements of a Purchase Agreement entered into by the Company with each of the institutional shareholders who received shares of Common Stock in connection with the Private Placement. The Registerable Shares are being offered by certain selling shareholders.

In connection therewith, as counsel to the Company, we have examined such corporate proceedings, records and documents as we considered necessary for the purposes of this opinion. We have relied upon certificates of officers of the Company where we have deemed it necessary in connection with our opinion.

For purposes of this opinion we have assumed the authenticity of all documents submitted to us as originals, the conformity of all documents submitted to us as copies and the authenticity of the originals of all documents submitted to us as copies. We have also assumed the legal capacity of all natural persons, the genuineness of the signatures of persons signing all

Board of Directors

December 15, 2015

Page 2

documents in connection with which this opinion is rendered, the authority of such persons signing on behalf of the parties thereto other than the Company, and the due authorization, execution and delivery of all documents by the parties thereto other than the Company. We have not independently established or verified any facts relevant to the opinions expressed herein, but have relied upon statements and representations of officers and other representatives of the Company, the selling shareholders and their agents.

Based upon and subject to the foregoing qualifications, we are of the opinion that the Registrable Shares have been duly authorized, validly issued, and fully paid and are nonassessable.

Our opinion expressed above is subject to the qualification that we express no opinion as to the applicability of, compliance with, or effect of any laws except the Virginia Stock Corporation Act, as amended (including statutory provisions, all applicable provisions of the Commonwealth of Virginia constitution and reported judicial decisions interpreting the foregoing).

We hereby consent to the filing of this opinion with the Commission as an exhibit to the Registration Statement and to the statements made in reference to our firm under the caption “Legal Matters” in the Prospectus which is a part of the Registration Statement. In giving this consent, we do not admit that we are in the category of the persons whose consent is required under Section 7 of the Act or the rules or regulations of the Commission.

We do not find it necessary for the purposes of this opinion, and accordingly we do not purport to cover in this opinion, the application of the securities or “Blue Sky” laws of the various states to the issuance and sale of the Registrable Shares.

This opinion is limited to the specific issues addressed herein, and no opinion may be inferred or implied beyond that expressly stated herein. This opinion speaks only as of the date that the Registration Statement becomes effective under the Act, and we assume no obligation to revise or supplement this opinion after the date of effectiveness of the Registration Statement should the Virginia Stock Corporation Act, as amended, be changed by legislative action, judicial decision or otherwise.

This opinion is furnished to you in connection with the filing of the Registration Statement and is not to be used, circulated, quoted or otherwise relied upon for any other purposes.

Very truly yours,

/s/ Edmunds & Williams, P.C.

EDMUNDS & WILLIAMS, P.C.