QuantumGPT
Physics is hard for AIs too
You may have noticed that OpenAI chat bot, ChatGPT is making a splash across the internet. Much of the commentary has centered around the vast attack surface of the chat bot, and the hijinks that can ensue when you ask it to plan the destruction of the human race as though it were a malevolent AI, or just a paperclip maximizer.
Others have been putting the bot through its paces by making it take the SAT1:
On quantum twitter results appear to be mixed. Verd gets a plausible looking answer, while Anastasia Marchenkova shows that GPT won’t be designing qubits any time soon.
After all of this excitement, I decided to get in on the game with some simple tasks for our friendly LLM. Does it know about SCQubits? Can it help me find the eigenenergies of a transmon? Will it solve my homework for me? Let’s find out!
Q1: Simulate a transmon
Prompt:
How would I use the SCQubits library to simulate the first 5 eigenenergies of a transmon with 50nA critical current and 100 fF shunt capacitance as a function of offset charge from 0 to 0.5?
Answer:
import scqubits as scq
import matplotlib.pyplot as plt
import numpy as np
transmon = scq.Transmon(EJ=50e-9, EC=100e-15, ng=0)
offset_charge_range = np.linspace(0, 0.5, 100)
eigenenergies = [transmon.eigenenergies(n=5, ng=offset_charge) for offset_charge in offset_charge_range]
for i in range(5):
plt.plot(offset_charge_range, eigenenergies[:,i], label=f"Eigenenergy {i+1}")
plt.xlabel("Offset Charge (a.u.)")
plt.ylabel("Eigenenergy (Hz)")
plt.legend()
plt.show()
Discussion: You’ll notice that this code block fails immediately, because the Transmon class requires an ncut definition. Additionally, it doesn’t really understand how transmon critical current and shunt capacitance translate to energy scales, so it just shoves the 50 nA and 100 fF values into the Ej and Ec parameters, respectively. Also, the correct method to use is get_spectrum_vs_paramvals().
BUT, recall that this is a chat bot, so you can just keep asking questions. I followed up with ‘What ncut should I use?’ . Here’s the answer:
You don’t need to be on the SCQubits dev team to know that the Transmon class has neither an energy_gap() nor an optimal_ncut() method.
OK, I’m going to revise my prompt.
Prompt:
How would I use the SCQubits library to simulate the first 5 eigenenergies of a transmon with 50nA critical current and 100 fF shunt capacitance as a function of offset charge from 0 to 0.5? Choose a reasonable value for ncut. Use the get_spectrum_vs_paramvals() method of the Transmon class.
Answer:
from scqubits import Transmon
from scqubits.utils.spectrum_utils import get_spectrum_vs_paramvals
transmon = Transmon(Ic=50e-9, Cshunt=100e-15)
ncut = 10
offset_charge_range = np.linspace(0, 0.5, 100)
eigenenergies = get_spectrum_vs_paramvals(transmon, 'offset_charge', offset_charge_range, ncut=ncut)Discussion:
Still the same mistakes, plus some new ones.
Verdict: ChatGPT clearly doesn’t truly understand the SCQubits documentation, even for trivial use cases like calling the Transmon class. Now I’m wondering whether Github CoPilot could do better.
Q2: Simulate CZ gate between 2 coupled fluxonia
Prompt:
Simulate the time evolution of a CZ gate between two Fluxonium qubits using QuTiP.
Answer:
The initial answer was wrong in a few ways, so I won’t take up space posting it. Here’s a list:
Used methods
iden()andcz()instead ofidentity()andcontrolled_gate()to make identity matrix and CZ interaction, respectively.Decided to calculate Pauli-X gate expectation values for each qubit instead of Z
Luckily, I was able to instruct it to fix these deficiencies, after which it produced code with only one weird bug (in bold, below) that I removed before being able to run it and get the expected output.
import numpy as np
import qutip as qt
# Define the initial state of the qubits
psi0 = qt.tensor(qt.basis(2,0), qt.basis(2,1))
# Define the Hamiltonian for the system
H = 0.5 * qt.tensor(qt.sigmax(), qt.identity(2)) + \
0.5 * qt.tensor(qt.identity(2), qt.sigmax()) + \
0.1 * qt.controlled_gate(qt.sigmaz())(2,0,1)
# Define the time points at which to calculate the evolution of the system
times = np.linspace(0, 10, 100)
# Use the mesolve function to simulate the time evolution of the system
result = qt.mesolve(H, psi0, times, [], [])
# Define the Pauli-Z operator for each qubit
sigmaz1 = qt.tensor(qt.sigmaz(), qt.identity(2))
sigmaz2 = qt.tensor(qt.identity(2), qt.sigmaz())
# Calculate the expectation value of the Pauli-Z operator for each qubit at each time point
expect1 = qt.expect(sigmaz1, result.states)
expect2 = qt.expect(sigmaz2, result.states)
# Visualize the results using matplotlib
import matplotlib.pyplot as plt
plt.plot(times, expect1, label="Qubit 1")
plt.plot(times, expect2, label="Qubit 2")
plt.xlabel("Time (s)")
plt.ylabel("Expectation value of Pauli-Z")
plt.legend()
plt.show()
Discussion:
Technically, ChatGPT was not able to produce working code. I think a user with little QuTIP experience might be able to muddle through here, because the bot was able to give me the correct functions after I pointed out that its choices weren’t working. Additionally, if the user knows which expectation value they want, the bot is able to trivially replace those parts of the code. The last bug, that weird tuple: (2,0,1) you can remove probably just by intuition.
Verdict:
I think I would give the bot a C on this one.
Q3: Solve my homework problems
I recently picked up Quantum Information & Quantum Optics with Superconducting Qubits by Juan José García Ripoll2 out of professional interest. Happily, the book comes with exercises at the end of each chapter, which will be useful both for students and me writing this blog post. Finding a reasonable problem was non-trivial. Many are graphical in nature, some quite involved. I did eventually decide on Exercise 4.4:
Prompt:
The energy of an LC circuit is given by
Prove that energy is conserved by showing that dH/dt = 0. Use the fact that:
Answer3:
We know that V and I are functions of time, so we can write V = V(t) and I = I(t). This allows us to rewrite H as:
We can now take the derivative of H with respect to time.
We can use the chain rule to rewrite the terms in this equation. The chain rule states that if
then
We know that V and I are functions of time, so we can rewrite the above equation as:
We can simplify this expression by combining the constants and canceling out the 2's:
Finally, we can plug in the expressions [above in the prompt]:
Simplifying this expression, we get:
Since I^2 and V^2 are always non-negative, we know that dH/dt is always non-positive. Therefore, dH/dt can only be 0 if I^2 = V^2, which means that I = +/- V.
Discussion:
Lol, lmao. ChatGPT was doing so well right up to that penultimate equation. Anyone reading this blog would be able to simplify the equation to get the correct equation:
ChatGPT, you really snatched defeat from the jaws of victory there.
Verdict:
If I were to put my TA hat back on again, I guess I would give this response partial credit, but with lots of red ‘????’s at the end trying to figure out how a student who has shown command of things like the chain rule has somehow failed to properly cancel terms. I’d probably chalk it up to dyslexia and see if they needed some sort of accommodation during tests or quizzes.
Conclusion:
Despite the hubbub ChatGPT has shown it clearly doesn’t understand what it is doing. It loses its train of ‘reasoning’ after a relatively short span for simple homework problems. Additionally, although it clearly knows about QuTIP and SCQubits it has failed to understand the documentation and, in the case of the Transmon, isn’t able to translate critical currents and capacitances into the relevant energy scales.
It is able to produce plausible looking output, but close inspection (and trying to run the code) shows really basic errors that often prevent execution.
Nonetheless, I think we’ve glimpsed something really tantalizing here, but I don’t know enough about LLMs to understand how close we are to useful technical AI assistants. Is it just a few steps to an LLM that ‘understands’ physics and math, or are we separated by an impassible chasm few meters wide but kilometers deep. Something that can’t be crossed without conceptually novel approaches to the problem.4
This is an American standardized test commonly used as a metric for University/College admission.
I kind of want to read the whole thing and review it, but also I can’t possibly do this in a reasonable amount of time? I am not sure the best way to tackle this. Perhaps just read the chapters of interest to me? Based on my skim through, it’s probably the best superconducting qubits book out there.
I’ve edited the response slightly where ChatGPT used LaTeX expressions for small equations that can be described in prose. Since Substack STILL doesn’t render LaTeX, I did this for readability.
Beatriz, if you’ve read this far, you’ve earned your shoutout.
But do you think that as this technology evolves there will be a no code interface for quantum machine learning that will be viable?