Hackathon Demonstration#
Beyond the two previous examples, which closely followed the Langchain custom agent tutorial, the goal for the final presentation was to highlight the use of LangSim in production. For this purpose, the functionality was transferred from the Jupyter notebook to a Python module. As a first step the LangSim module is loaded using the %reload_ext jupyter magic command:
%reload_ext langsim
Then the Open AI API key is set to the OPENAI_API_KEY environment variable so it can be used by LangSim internally:
import os, getpass
os.environ['OPENAI_API_KEY'] = getpass.getpass("Enter your API key")
Dialog#
From this point on the user can chat with the LangSim interface using the chat jupyter magic. As a first challenge the user communicates their interest to find a material with a bulk modulus in the range of 200 GPa; following with a clarification of this statement by asking the LLM to calculate the bulk modulus for the noble metals. In contrast to the previous example, the LLM is tasked with repeating the same calculation multiple times and then identify the chemical element, which most closely matches the targeted bulk modulus of 200GPa. None of the required logic is implemented explicitly, rather the LLM only has access to a get_equilibrium_lattice() function to calculate the equilibrium crystal structure and a get_bulk_modulus() function, which calculates the bulk modulus for a given equilibrium crystal structure.
%%chat
I am looking for a material with a bulk modulus in the range of 200 GPa.
Can you compute the bulk modulus for the noble metals for me?
> Entering new AgentExecutor chain...
Sure, I can help with that. The noble metals include gold (Au), silver (Ag), and platinum (Pt). Which model would you like to use for the calculations: EMT or MACE?
> Finished chain.
Sure, I can help with that. The noble metals include gold (Au), silver (Ag), and platinum (Pt). Which model would you like to use for the calculations: EMT or MACE?
The LLM is instructed by the system prompt to clarify which simulation method it should use to calculate a given material property. At the time of the demonstration, two simulation methods were implemented, EMT and MACE. As the user in this demonstration is not aware of the technical details of these simulation methods, he asks for clarification. This highlights how LLMs can use the context of the previous conversation to answer a given question:
%%chat
What are those methods?
> Entering new AgentExecutor chain...
Certainly! Here is a brief overview of the two models:
1. **Effective Medium Theory (EMT)**:
- **Description**: EMT is a computationally efficient, analytical model that describes the macroscopic properties of composite materials.
- **Advantages**: It is relatively fast and can provide good approximations for certain types of materials.
- **Limitations**: It may not capture all the complexities of atomic interactions, especially for materials with complex bonding.
2. **MACE (Machine Learning Force Field)**:
- **Description**: MACE is a machine learning force field for predicting many-body atomic interactions that covers the periodic table.
- **Advantages**: It leverages machine learning to provide highly accurate predictions of atomic interactions and material properties.
- **Limitations**: It can be computationally more intensive compared to EMT.
Please choose one of these models for the calculations.
> Finished chain.
Certainly! Here is a brief overview of the two models:
Effective Medium Theory (EMT):
Description: EMT is a computationally efficient, analytical model that describes the macroscopic properties of composite materials.
Advantages: It is relatively fast and can provide good approximations for certain types of materials.
Limitations: It may not capture all the complexities of atomic interactions, especially for materials with complex bonding.
MACE (Machine Learning Force Field):
Description: MACE is a machine learning force field for predicting many-body atomic interactions that covers the periodic table.
Advantages: It leverages machine learning to provide highly accurate predictions of atomic interactions and material properties.
Limitations: It can be computationally more intensive compared to EMT.
Please choose one of these models for the calculations.
Based on the explanation of the LLM, the user chooses the EMT method, primarily because it is computationally more efficient to calculate a large number of calculations in a short amount of time. The LLM then executes two calculations for each of the three noble metals silver (Ag), gold (Au) and platinum (Pt). These calculations are the calculation of the equilibrium structure get_equilibrium_lattice() which in addition to the chemical symbol now also takes the name of the chosen calculator as an input and the calculation of the bulk modulus get_bulk_modulus() which takes the optimized structure as AtomsDict() object as an input in addition to the chosen calculator as string.
%%chat
Let's take EMT then
> Entering new AgentExecutor chain...
Invoking: `get_atom_dict_bulk_structure` with `{'chemical_symbol': 'Au'}`
numbers=[79] positions=[[0.0, 0.0, 0.0]] cell=[[0.0, 2.04, 2.04], [2.04, 0.0, 2.04], [2.04, 2.04, 0.0]] pbc=[True, True, True]
Invoking: `get_atom_dict_bulk_structure` with `{'chemical_symbol': 'Ag'}`
numbers=[47] positions=[[0.0, 0.0, 0.0]] cell=[[0.0, 2.045, 2.045], [2.045, 0.0, 2.045], [2.045, 2.045, 0.0]] pbc=[True, True, True]
Invoking: `get_atom_dict_bulk_structure` with `{'chemical_symbol': 'Pt'}`
numbers=[78] positions=[[0.0, 0.0, 0.0]] cell=[[0.0, 1.96, 1.96], [1.96, 0.0, 1.96], [1.96, 1.96, 0.0]] pbc=[True, True, True]
Invoking: `get_atom_dict_equilibrated_structure` with `{'atom_dict': {'numbers': [79], 'positions': [[0.0, 0.0, 0.0]], 'cell': [[0.0, 2.04, 2.04], [2.04, 0.0, 2.04], [2.04, 2.04, 0.0]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
Step Time Energy fmax
LBFGS: 0 19:41:36 0.002606 0.308859
LBFGS: 1 19:41:36 0.000032 0.077808
LBFGS: 2 19:41:36 -0.000135 0.003099
LBFGS: 3 19:41:36 -0.000135 0.000029
LBFGS: 4 19:41:36 -0.000135 0.000000
numbers=[79] positions=[[4.761270571021482e-17, -3.4431732128676475e-17, -2.0729599738876008e-16]] cell=[[7.040904860568554e-17, 2.028082809705617, 2.0280828097056176], [2.028082809705617, 1.0384771021574885e-16, 2.0280828097056176], [2.028082809705617, 2.028082809705618, 4.963320342464553e-17]] pbc=[True, True, True]
Invoking: `get_atom_dict_equilibrated_structure` with `{'atom_dict': {'numbers': [47], 'positions': [[0.0, 0.0, 0.0]], 'cell': [[0.0, 2.045, 2.045], [2.045, 0.0, 2.045], [2.045, 2.045, 0.0]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
Step Time Energy fmax
LBFGS: 0 19:41:36 0.001584 0.198735
LBFGS: 1 19:41:36 0.000254 0.113272
LBFGS: 2 19:41:36 -0.000366 0.004325
LBFGS: 3 19:41:36 -0.000367 0.000089
LBFGS: 4 19:41:36 -0.000367 0.000000
numbers=[47] positions=[[3.632034897593557e-17, -7.643082441891503e-17, -4.3481411845850515e-16]] cell=[[9.839176801279394e-17, 2.031776305380803, 2.0317763053808036], [2.031776305380803, -6.293048194449085e-17, 2.0317763053808036], [2.031776305380803, 2.031776305380803, -2.945414531846451e-17]] pbc=[True, True, True]
Invoking: `get_atom_dict_equilibrated_structure` with `{'atom_dict': {'numbers': [78], 'positions': [[0.0, 0.0, 0.0]], 'cell': [[0.0, 1.96, 1.96], [1.96, 0.0, 1.96], [1.96, 1.96, 0.0]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
Step Time Energy fmax
LBFGS: 0 19:41:36 -0.000125 0.036183
LBFGS: 1 19:41:36 -0.000149 0.004500
LBFGS: 2 19:41:36 -0.000150 0.000013
LBFGS: 3 19:41:36 -0.000150 0.000000
numbers=[78] positions=[[1.733508800950033e-17, 1.1611829961508788e-16, 1.1092203033984378e-16]] cell=[[4.60232654901218e-18, 1.9609007393798525, 1.960900739379854], [1.9609007393798532, -3.193849078504324e-16, 1.9609007393798543], [1.9609007393798532, 1.9609007393798528, 1.704915243852484e-16]] pbc=[True, True, True]
Invoking: `get_bulk_modulus` with `{'atom_dict': {'numbers': [79], 'positions': [[4.761270571021482e-17, -3.4431732128676475e-17, -2.0729599738876008e-16]], 'cell': [[7.040904860568554e-17, 2.028082809705617, 2.0280828097056176], [2.028082809705617, 1.0384771021574885e-16, 2.0280828097056176], [2.028082809705617, 2.028082809705618, 4.963320342464553e-17]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
173.83830457892148
Invoking: `get_bulk_modulus` with `{'atom_dict': {'numbers': [47], 'positions': [[3.632034897593557e-17, -7.643082441891503e-17, -4.3481411845850515e-16]], 'cell': [[9.839176801279394e-17, 2.031776305380803, 2.0317763053808036], [2.031776305380803, -6.293048194449085e-17, 2.0317763053808036], [2.031776305380803, 2.031776305380803, -2.945414531846451e-17]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
100.1580099631827
Invoking: `get_bulk_modulus` with `{'atom_dict': {'numbers': [78], 'positions': [[1.733508800950033e-17, 1.1611829961508788e-16, 1.1092203033984378e-16]], 'cell': [[4.60232654901218e-18, 1.9609007393798525, 1.960900739379854], [1.9609007393798532, -3.193849078504324e-16, 1.9609007393798543], [1.9609007393798532, 1.9609007393798528, 1.704915243852484e-16]], 'pbc': [True, True, True]}, 'calculator_str': 'emt'}`
278.29661366411915Here are the calculated bulk moduli for the noble metals using the EMT model:
- **Gold (Au)**: 173.84 GPa
- **Silver (Ag)**: 100.16 GPa
- **Platinum (Pt)**: 278.30 GPa
Among these, Platinum (Pt) has a bulk modulus within the range of 200 GPa.
> Finished chain.
Here are the calculated bulk moduli for the noble metals using the EMT model:
Gold (Au): 173.84 GPa
Silver (Ag): 100.16 GPa
Platinum (Pt): 278.30 GPa
Among these, Platinum (Pt) has a bulk modulus within the range of 200 GPa.
The results are summarized in a short list. This demonstration highlights that LangSim can be used without prior technical experience in calculating material properties. From students to teachers and professors, it is a handy tool for everybody who wants to have quick access to material properties computed with one of the standard methods like EMT. Obviously, the MACE method is more accurate than the EMT method, but also computationally more expensive.
Still, to further highlight the real-life application of a tool like LangSim the next step after the completion of these calculations is the comparison to an experimental reference. For this purpose, the get_experimental_elastic_property_wikipedia() function is implemented which downloads the experimental values for the elastic properties from the wikipedia site for elastic properties.
%%chat
How does this compare to experiment?
> Entering new AgentExecutor chain...
Invoking: `get_experimental_elastic_property_wikipedia` with `{'chemical_symbol': 'Au'}`
{'youngs_modulus': 78.0, 'poissons_ratio': 0.44, 'bulk_modulus': 180.0, 'shear_modulus': 27.0}
Invoking: `get_experimental_elastic_property_wikipedia` with `{'chemical_symbol': 'Ag'}`
{'youngs_modulus': 83.0, 'poissons_ratio': 0.37, 'bulk_modulus': 100.0, 'shear_modulus': 30.0}
Invoking: `get_experimental_elastic_property_wikipedia` with `{'chemical_symbol': 'Pt'}`
{'youngs_modulus': 168.0, 'poissons_ratio': 0.38, 'bulk_modulus': 230.0, 'shear_modulus': 61.0}Here are the experimental bulk moduli for the noble metals compared to the calculated values using the EMT model:
- **Gold (Au)**:
- **Calculated (EMT)**: 173.84 GPa
- **Experimental**: 180.0 GPa
- **Silver (Ag)**:
- **Calculated (EMT)**: 100.16 GPa
- **Experimental**: 100.0 GPa
- **Platinum (Pt)**:
- **Calculated (EMT)**: 278.30 GPa
- **Experimental**: 230.0 GPa
The calculated values are quite close to the experimental values, with the exception of Platinum (Pt), which shows a larger discrepancy.
> Finished chain.
Here are the experimental bulk moduli for the noble metals compared to the calculated values using the EMT model:
Gold (Au):
Calculated (EMT): 173.84 GPa
Experimental: 180.0 GPa
Silver (Ag):
Calculated (EMT): 100.16 GPa
Experimental: 100.0 GPa
Platinum (Pt):
Calculated (EMT): 278.30 GPa
Experimental: 230.0 GPa
The calculated values are quite close to the experimental values, with the exception of Platinum (Pt), which shows a larger discrepancy.
Finally, as the last step the user tasks the LLM to compare the theoretical predictions and the experimental results to give an indication of the trustworthiness of these calculations. Given the optimization for computational efficiency, the result for Platinum with an error of over 20% is insufficient for any real life application. Still, this can be addressed by switching to the MACE model.
%%chat
Can you show me the deviation in percent?
> Entering new AgentExecutor chain...
Certainly! Let's calculate the percentage deviation between the calculated and experimental bulk moduli for the noble metals.
The formula for percentage deviation is:
\[ \text{Percentage Deviation} = \left( \frac{\text{Calculated Value} - \text{Experimental Value}}{\text{Experimental Value}} \right) \times 100 \]
Here are the calculations:
1. **Gold (Au)**:
- Calculated (EMT): 173.84 GPa
- Experimental: 180.0 GPa
- Percentage Deviation: \[ \left( \frac{173.84 - 180.0}{180.0} \right) \times 100 \approx -3.42\% \]
2. **Silver (Ag)**:
- Calculated (EMT): 100.16 GPa
- Experimental: 100.0 GPa
- Percentage Deviation: \[ \left( \frac{100.16 - 100.0}{100.0} \right) \times 100 \approx 0.16\% \]
3. **Platinum (Pt)**:
- Calculated (EMT): 278.30 GPa
- Experimental: 230.0 GPa
- Percentage Deviation: \[ \left( \frac{278.30 - 230.0}{230.0} \right) \times 100 \approx 21.87\% \]
So, the percentage deviations are:
- **Gold (Au)**: -3.42%
- **Silver (Ag)**: 0.16%
- **Platinum (Pt)**: 21.87%
These deviations indicate how close the calculated values are to the experimental values.
> Finished chain.
Certainly! Let’s calculate the percentage deviation between the calculated and experimental bulk moduli for the noble metals.
The formula for percentage deviation is: [ \text{Percentage Deviation} = \left( \frac{\text{Calculated Value} - \text{Experimental Value}}{\text{Experimental Value}} \right) \times 100 ]
Here are the calculations:
Gold (Au):
Calculated (EMT): 173.84 GPa
Experimental: 180.0 GPa
Percentage Deviation: [ \left( \frac{173.84 - 180.0}{180.0} \right) \times 100 \approx -3.42% ]
Silver (Ag):
Calculated (EMT): 100.16 GPa
Experimental: 100.0 GPa
Percentage Deviation: [ \left( \frac{100.16 - 100.0}{100.0} \right) \times 100 \approx 0.16% ]
Platinum (Pt):
Calculated (EMT): 278.30 GPa
Experimental: 230.0 GPa
Percentage Deviation: [ \left( \frac{278.30 - 230.0}{230.0} \right) \times 100 \approx 21.87% ]
So, the percentage deviations are:
Gold (Au): -3.42%
Silver (Ag): 0.16%
Platinum (Pt): 21.87%
These deviations indicate how close the calculated values are to the experimental values.
Summary#
The jury was convinced that this demonstration of the capabilities of LangSim highlights a future direction in the application of LLMs for materials and chemistry. So the LangSim team won the first prize sponsored by RadicalAI.
In comparison to the previous examples, the highlight of this presentation, besides hiding the technical complexity by developing a python module, was the capability of the LLM to repeat the workflow consisting of two calculations, the get_equilibrium_lattice() function and the get_bulk_modulus() function, over three elements without the need to define any additional functionality.