I found myself reading Dan Luu’s recent piece on Productivity and Velocity, and then re-read 95%-ile isn’t that good. This led me to wonder how I stack up in the world of professional physicists. Am I in the 95th percentile? If not, who is? How can I get there?
Reader, down this path lies madness.
A Few Good Physicists
Stack ranking physicists in a general sense is basically pointless. The main problem is that being a professional physicist requires two things:
A good command/mastery of some field(s) of physics
Some other activity (teaching, researching, etc)
#2 is important here. Physicists cannot just know physics. They must do it. It is not sufficient to work through all 10 volumes of Landau and Lifshitz. You’ve gotta do something after you’re done!
Usually this something is research, teaching, or some combination of the two. But there are many different research settings, career stages, long-term goals, etc, that can affect your measure of goodness. Many reductive, non-comparable metrics exist: Accolades (Nobel prize, other awards), h-index, student evaluations, # of students graduated, and more.
So, I assert that asking yourself ‘how do I rank against other physicists?’ is fundamentally unanswerable and will lead you to madness.
Let us ask a better question, instead.
How Can I Become a Better Physicist?
This question also turns out to be hard to answer in a simple way, but it might actually be answerable. In my mind, being a better physicist has two components:
Personal- This is includes breadth/depth of knowledge in my field(s), technical ability (familiarity with new techniques, refining old techniques), clarity of thought, etc
Contextual- How can I use my physics powers to improve outcomes in my organization? ‘Organization’ here should be interpreted loosely. It could be a graduate research group, an engineering team, an international collaboration, and so on.
Personal Improvement
As Dan Luu observed in his 95%-ile post, it is much, much easier to improve in activities objective measures and outcomes. Usually these are games. Watch game film, get a coach, get people much better than you to watch you play and offer advice. Gigantic communities of players exist out there and many top players are happy to offer insight and advice.
Frustratingly, the physics community seems much more limited. Yes, there exists a Physics Stack Exchange, but the questions seem to be much more appropriate for answer questions for undergraduates than providing insight at the professional level.1 Indeed, it seems like the majority of resources for systematic practical physics improvement exist at the undergraduate level.
As an undergrad, or even a grad student, it is possible to submit ‘game film’ (an attempt at a physics problem) to a professional (your TA, your professor, your advisor) and get feedback about where you went wrong and what to do differently.
Unfortunately, ‘game film’ soon becomes, almost exclusively, either arXiv pre-prints or papers from physics journals. The newer the paper, the more limited the pool of people who can comment intelligently becomes. Sometimes the pool of experts does not extend beyond the paper’s authors. It should be easier for us to communicate our work to our peers, and it should be easier for them to understand what we’ve done.
Sadly, the act of authoring a paper soon becomes a tedious, Sisyphean slog through endless drafts and Reviewer 2’s mean comments. And, of course, the experience of reading papers is strongly dependent on how well they’re written.2
Reading Papers
For the foreseeable future reading and writing papers will suck. Nonetheless, it is absolutely critical to keep reading papers. Even if it’s only to build a network of concepts and previous work in your mind. This helps to understand the state of the field and contextualize/motivate not only your own work, but those of others.
Since scientific papers are generally difficult to read, and there are so many of them, it is critically important that you also get better at both filtering and reading. A method that I think is close to optimal is reading in this order:
Abstract
Conclusion
Figures
Everything Else (optional)
Steps 1-3 here are filtering steps. I often stop after the abstract, for example. If it seems worthwhile to go beyond Step 3, then I prioritize what is most important to me at the time. It could be Methods, it could be Results, or it could be the motivation of the paper (usually the Introduction).
I am not sure if a more optimal method exists, but I get the sense that most other people follow a procedure similar to my own.
The much trickier problem is Knowledge Management. I’m sure gigabytes of text have been written on the subject, dozens of Knowledge Management Systems exist to help you store, contextualize, and cross-reference papers and other media. I just load papers into Notability and read them on a tablet, and write notes in the margins. The act of writing notes helps me remember that I’ve read a paper on $TOPIC or $METHOD when it’s relevant.
Writing (Papers?)
I think it is universally acknowledged that writing is a critically important skill in basically every profession. The same is true in physics, whether it is writing a paper, proposal, or a simple ‘how-to’ document. I’ve always found that even the act of writing a simple outline for technical document immediately illuminates massive gaps where I had assumed there had been actual knowledge.
Happily, writing is one of those activities where feedback can be rapid, so at least you can quickly determine whether you’re communicating effectively. It is less easy to pinpoint why exactly some passage may be unclear, or what to do to fix it. I recommend having one or more friends who are (semi)professional writers and willing to help you3. Otherwise, just grind it out, I guess?
Lastly, if you write extensively already, it may be worth working on writing faster4.
Trying the Thing
One thing that many experimentalists seem to have in common is that grad school was the last time we did any ‘serious’ calculation in pursuit of a physics result. It’s so easy to get wrapped up in debugging experiments, catching up with team-members, helping them debug experiments, writing up results, that you can avoid doing ‘theory’ work for months or years!
Building new models or reproducing published simulations can be eye-opening, educational, and fun! It also instills a renewed respect for the important, but oft-overlooked nitty-gritty details of how and why our devices function the way they do. On more than one occasion, building my own model of an experiment helped me better understand how it should be analyzed and make changes to what we thought were well-established protocols.
Next time you see a novel qubit system described in a paper, try reproducing some of the figures in the paper. It may give you insight into tricky details that the authors may have omitted. This is easier than ever with the proliferation of nice open source tools made exactly for this purpose.
If you’re a theorist, you probably do this kind of stuff all the time. Or, at least, whenever you’re not searching for bugs in your code, or learning more about numerical recipes. In that case, I suggest you stroll down to the lab that houses the experimental group you support/collaborate with (you DO collaborate with experimentalists, right?) and ask them to walk you through a typical measurement. Do this regularly.
Meet Good Physicists
This is a hybrid of personal and contextual improvement. I could have put this in the next section and called it “Hire Good Physicists”.
Getting good at meeting people is definitely a skill, one that I’m not great at. Conferences are a nice opportunity to meet people you admire, especially if you see interesting work and room for collaboration! Of course you already know this in your heart, but networking is nerve-wracking, scary, and sometimes can feel ‘gross’.
My solution to this problem in QC was to get hired into a company that employs many excellent physicists and learn from them. Since that time, there are more high quality QC companies than ever, and they’re always looking for good people. As usual, it helps to already know someone in your target company, otherwise you can do what I did and blast an incredible quantity of applications into the ether.
Since then, I’ve benefited immensely from the expertise and careful thinking of my coworkers. My pro-tip for maximizing this benefit is to start by asking relatively specific questions (this is actually good advice at any stage of your career). This means doing some pre-work trying to get to grips with the problem you’re having. Then, once you've got ‘em hooked with your initial question, broaden the scope with follow-ups and maybe you can get some much more generalized wisdom. Also, don’t forget to write down the answers!
Organizational Improvement
An important insight in Dan’s 95%-ile post is that the differentiator between 95th percentile and 99.5th percentile Overwatch players is not only mechanical skill, but also the incidence rate of bone-headed game-losing mistakes. 50th and 95th percentile players commit many more blunders than pro-tier players. At the highest levels, one or two such errors can be game-losers, while even at the 95th percentile, a team could commit a dozen blunders and still win.
In the context of professional physics, you want to think about where you can use your powers to prevent your organization from committing dumb mistakes. Here are some I’ve seen/perpetrated5:
Design errors that make entire fabrication runs unusable.
Code bugs that corrupt long-term (overnight or multi-day) data collection, rendering the data worthless.
Experimental design errors that make entire fabrication runs unusable even when the chip layout is perfect.
Analysis errors that vastly change the perceived outcome of an experiment, driving downstream decision making in the wrong direction.
These are worst case outcomes, but even relatively mild forms of these mistakes can waste days, weeks, or months of time. I’m sure these mistakes, or analogues of them are very familiar to pretty much anyone reading this blog. Happily, these particular issues can be “easily” solved with some forethought and decent process.
As you reduce the incidence rate of the most obvious failures in your organization, you will get visibility into the next layer, then the next, and so on. You can help build institutional knowledge and operational competence that will long outlast your frail corporeal vessel.
As you think about more ways in which you can help your team, you should prefer actions which help everyone more easily do the right thing. Your interventions should make good process much easier to execute than bad. This won’t guarantee good physics, but it should tip the odds in your favor.
How Do I Stack Up?
I’ve spilled many bits about how to be better, and if I’m so wise and knowledgable, why aren’t I CEO of my own multibillion dollar quantum company? Or maybe I am! (I’m not). I’m just a regular, sentient, unblinking Bohr atom like the rest of you.
Reading: It’s often hard to justify spending 1+ hours a day sifting through new arXiv pre-prints when there is SO MUCH WORK to do. Honestly, I usually feel pretty overwhelmed by the sheer volume of new arXiv submissions every day. Nonetheless, keeping tabs on the field is work, and it should be done. I must do better.
Writing: Well, you’ve made it this far so it can’t be all bad. Authoring this blog has helped me organize vague ideas I’ve had for a while, and I manage to write a reasonable amount during the work day as well, including some papers and internal communications over the past year or so. I could definitely do more here, and I’m particularly interested in improving my persuasive physics writing.
Trying the Thing: This is my weakest area, by far. But I have noticed that I engage with more new physics when I am in periods of high arXiv readership vs when I am relatively unengaged with the literature. It should be relatively easy to improve here, there are now so many tools available to engage with quantum systems, that it’s hard NOT to accidentally simulate a qubit.
Meeting Good Physicists: I did a good job getting hired into my physics organization, but a not a great job of meeting people at conferences and striking up conversations about their work. I’m planning to do better at APS March Meeting 2022.
Organizational Improvement: I’ll avoid commenting with any specificity about how my org stacks up. I think it’s generally much easier to identify areas for improvement on an organizational level. Probably because it’s always easier to identify external flaws than internal ones. Actually implementing improvements, well, that can be hard.
Final Thoughts
Some of our software tools for doing physics have become substantially better and widely proliferated. They’re open source, have growing communities, and are actively supported. This is cool and good.
On the other hand, some of our most important tools for disseminating new knowledge and getting feedback are not equal to the task6. Similarly, knowledge of finicky experimental/analytical tricks is almost exclusively passed down via the oral tradition, which is why it helps to know as many good physicists as possible. If you don’t know the right person, you end up rediscovering ‘known’ techniques, usually with great expenditure of blood, sweat, and tears.
It doesn’t have to be that way.
If you’ve made it this far, why not subscribe? You get my updates into your inbox, and I get to see Number Go Up!
This may be an unfair characterization, largely because I don’t use the Physics Stack Exchange. Am I wrong?
How is it that this critical tool in our arsenal can still be so limited in the year 2021? How can we improve both the production and consumption of our ‘game film’?
These questions and their answers deserve their post, or even a series of posts. My personal wishlist is this:
Plots/figures should be interactive.
Data and analysis code should be included by default.
The paper itself should walk the reader through the data to the conclusions the authors have reached.
The paper should be written in plain language and spurn jargon.
Methods sections should be step-by-step instructions.
A PDF should be generated as a document of last resort for cases where no internet access/paper reading software is available.
I solicited actual writing feedback for this post. Any shitty writing is purely a result of my inability to effectively act on my proof readers’ suggestions.
See Productivity and Velocity re: the benefits of writing faster.
If you think you’ve never done something like this, you may be the single most dangerous person in your lab.
As amazing as it is, the arXiv is essentially a rocket powered horse. Why did we put rockets on the horse instead of building the rest of the jet?