TL;DR, I throw a bunch of molecules at a pile of linear algebra, and hope predicted values line up with known experimental values; then I use the pile of linear algebra on novel molecules.
There’s a bit more to it than that, like how to represent molecules in a computer-readable format, generating additional input variables (molecular characteristics), input variable down-selection and/or dimensionality reduction, the specific ML models we use (feed-forward MLPs and graph convolution nets), and how to interpret results as they relate back to combustion.
From a broad perspective, our work is just a small part of a larger push from the Department of Energy to find economically-viable alternative liquid fuels. ML speeds up the process of screening candidate molecules, for example those found in bio-oil resulting from pyrolizing and catalytically-upgrading lignocellulosic biomass or other renewable sources. Our colleagues don’t have to synthesize large samples of many molecules just to test their properties and determine how they will behave in existing engines (a very costly and time-consuming process), instead we predict the properties and behaviors to highlight viable candidates so our colleagues can focus on analyzing those.
These papers (1, 2, 3) best outline the procedures and motivations for this work. PM me if you can’t get access and I’ll send you them!
cool, i’ll check out those papers.
have you had much success in discovering new molecules this way? it seems like if it works then that means that the neural net that emerged has discovered some law of physics (or property of chemistry, w/e) that we do not know. in other words, within the sequence of calculations lies some physical law.
Sort of - the models are able to predict numerical property values given a large amount of data to observe during training. In other words, given the scope of known data, we can extrapolate predictions for new data. The predictive capabilities of the model are only as reliable as the data used to train it, and unfortunately in our case we only have hundreds of samples per property, as opposed to other ML tasks with millions of samples. This highlights how much time it actually takes to find, synthesize, and experimentally test molecules!
Unfortunately neural networks, especially traditional multi-layered feed-forward networks, are often seen as a “black box” approach to regression and classification, where we don’t really understand how a network learns or why its weights are tuned the way they are. Analysis methods have come a long way, but ambiguity still exists.
What we have done, however, is find the statistical significance of specific molecular substructures as they relate to combustion properties. For example, when we trained our models to predict sooting propensity (amount of pollution formed during combustion), we noticed that various algorithms such as random forest regression were putting a heck of a lot more weight into a molecular variable measuring path length (length of carbon chains, number of higher order bonds); from this, we were able to conclude that long-chain hydrocarbons with a higher number of double or triple bonds form more soot, and an idea of what mechanistic pathways we should stay away from when producing bio-oil.
As for fuel-grade molecules, we’ve found that furanic compounds and compounds with cyclohexane substructures generally have equal operating efficiency (cetane number), equal energy density (lower heating value, MJ/kg), operate well in various environments (optimal flash, boiling, and cloud points, deg. C), all while producing much less soot (yield sooting index) compared to diesel fuel. The next step is finding a cheap way to mass produce the stuff!
Recently we’ve started down the rabbit hole of fungus-derived bio-oils, terpenes (yes, those terpenes!) derived from fungus may be useful for use as soot-reducing fuel additives.
this is actually very interesting.
i take it you’ve heard of the concept of “mechanistic interpretability”? perhaps you could learn something about your networks by implementing some of that methodology. here’s a glossary. also recommend poking around neelanda’s blog if you want to learn more.
interesting. how does that process work?
TL;DR, I throw a bunch of molecules at a pile of linear algebra, and hope predicted values line up with known experimental values; then I use the pile of linear algebra on novel molecules.
There’s a bit more to it than that, like how to represent molecules in a computer-readable format, generating additional input variables (molecular characteristics), input variable down-selection and/or dimensionality reduction, the specific ML models we use (feed-forward MLPs and graph convolution nets), and how to interpret results as they relate back to combustion.
From a broad perspective, our work is just a small part of a larger push from the Department of Energy to find economically-viable alternative liquid fuels. ML speeds up the process of screening candidate molecules, for example those found in bio-oil resulting from pyrolizing and catalytically-upgrading lignocellulosic biomass or other renewable sources. Our colleagues don’t have to synthesize large samples of many molecules just to test their properties and determine how they will behave in existing engines (a very costly and time-consuming process), instead we predict the properties and behaviors to highlight viable candidates so our colleagues can focus on analyzing those.
These papers (1, 2, 3) best outline the procedures and motivations for this work. PM me if you can’t get access and I’ll send you them!
cool, i’ll check out those papers. have you had much success in discovering new molecules this way? it seems like if it works then that means that the neural net that emerged has discovered some law of physics (or property of chemistry, w/e) that we do not know. in other words, within the sequence of calculations lies some physical law.
Sort of - the models are able to predict numerical property values given a large amount of data to observe during training. In other words, given the scope of known data, we can extrapolate predictions for new data. The predictive capabilities of the model are only as reliable as the data used to train it, and unfortunately in our case we only have hundreds of samples per property, as opposed to other ML tasks with millions of samples. This highlights how much time it actually takes to find, synthesize, and experimentally test molecules!
Unfortunately neural networks, especially traditional multi-layered feed-forward networks, are often seen as a “black box” approach to regression and classification, where we don’t really understand how a network learns or why its weights are tuned the way they are. Analysis methods have come a long way, but ambiguity still exists.
What we have done, however, is find the statistical significance of specific molecular substructures as they relate to combustion properties. For example, when we trained our models to predict sooting propensity (amount of pollution formed during combustion), we noticed that various algorithms such as random forest regression were putting a heck of a lot more weight into a molecular variable measuring path length (length of carbon chains, number of higher order bonds); from this, we were able to conclude that long-chain hydrocarbons with a higher number of double or triple bonds form more soot, and an idea of what mechanistic pathways we should stay away from when producing bio-oil.
As for fuel-grade molecules, we’ve found that furanic compounds and compounds with cyclohexane substructures generally have equal operating efficiency (cetane number), equal energy density (lower heating value, MJ/kg), operate well in various environments (optimal flash, boiling, and cloud points, deg. C), all while producing much less soot (yield sooting index) compared to diesel fuel. The next step is finding a cheap way to mass produce the stuff!
Recently we’ve started down the rabbit hole of fungus-derived bio-oils, terpenes (yes, those terpenes!) derived from fungus may be useful for use as soot-reducing fuel additives.
this is actually very interesting. i take it you’ve heard of the concept of “mechanistic interpretability”? perhaps you could learn something about your networks by implementing some of that methodology. here’s a glossary. also recommend poking around neelanda’s blog if you want to learn more.
Thanks for sharing! These seem to focus on LLMs/transformers, but since they use MLPs I should be able to find a way to adapt them for my use!