An E-104 is simply a document that communicates to the German system that you were in fact covered by another public health care provider somewhere in the EU.
To get one, you must contact your previous healthcare provider and ask them for one. Keep in mind that they will need to fill it out.
Link to the E-104 template in English below; for E104s in all other EU languages click here.
Addition: E-104 has been replaced by EESSI, and there is no need to request E-104 anymore. You can find out more information on EESSI here.