Benefiting from multi-energy X-ray imaging technology, material decomposition facilitates the differentiation of different materials in X-ray imaging. However, the performance of material decomposition is limited by the accuracy of the decomposition model. Due to the presence of non-ideal effects in imaging systems, it is difficult to explicitly build the imaging system models for material decomposition. We propose a novel machine learning-based pipeline to perform material decomposition using machine learning algorithms. Feature extraction is involved into the pipeline to improve the performance of material decomposition.