About Me

Hello! I am a Ph.D. candidate in Computer Science at Carnegie Mellon Univerity, advised by Zico Kolter .

My research currently focuses on understanding generative image models through the lens of prompt discovery. By finding prompts that induce specific, undesired behaviors from the models, we may gain a better understanding of the edge cases of our models and the gaps in our training data.

Previously, I have worked on understanding the long-term social ramifications of data-driven decision systems, especially as they relate to the criminal justice system. My work highlighted ways to engage stakeholders and use their expertise in determining the effectiveness of ML algorithms.

Before CMU, I was a Post-Baccalaureatte Research Fellow with the Beckman Laser Institute at the University of California, Irvine, where I worked on automated analyses of in-vivo multi-photon microscopy images. Even earlier, I was an undergraduate at Hampton University in the Mathematics Department.

Publications, Preprints, and Workshop Papers

( * Equal Contribution )

  • Rethinking Distance Metrics for Counterfactual Explainability
    Joshua Nathaniel Williams, Anurag Katakkar, Hoda Heidari, J Zico Kolter
    Arxiv Preprint [paper]  

  • Prompt Recovery for Image Generation Models: A Comparative Study of Discrete Optimizers
    Joshua Nathaniel Williams, Avi Schwarzschild, J Zico Kolter
    Arxiv Preprint [paper]  

  • FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
    Joshua Nathaniel Williams, J Zico Kolter
    2024 Conference on Language Modeling (COLM 2024) [paper] [data]  

  • DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted Image Generation
    Joshua N Williams, Molly FitzMorris, Osman Aka, Sarah Laszlo
    Arxiv Preprint [paper] [data]  

  • Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation
    Yutong He, Alexander Robey, Naoki Murata, Yiding Jiang, Joshua Williams, George J Pappas, Hamed Hassani, Yuki Mitsufuji, Ruslan Salakhutdinov, J Zico Kolter
    Arxiv Preprint [paper]  

  • Safe Use of Machine Learning for Air Force Human Resource Management
    Joshua Snoke, Matthew Walsh, Joshua Williams, David Schulker
    The RAND Corporation [paper]  

  • Local justice and machine learning: Modeling and inferring dynamic ethical preferences toward allocations
    Violet Xinying Chen, Joshua Williams, Derek Leben, Hoda Heidari
    2023 Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2023) [paper]  

  • A Bayesian Model of Cash Bail Decisions
    Joshua Williams, J. Zico Kolter
    2021 ACM Conference on Fairness, Accountability, and Transparency (ACM FAcct 2021)
    [paper] [data]  

  • A Call for Universities to Develop Requirements for Community Engagement in AI Research
    Emily Black*, Joshua Williams*, Michael Madaio, Priya L Donti
    Fair & Responsible AI Workshop @ Chi2020
    [paper]  

  • Dynamic Modeling and Equilibria in Fair Decision Making
    Joshua Williams, J. Zico Kolter
    NeurIPS 2019 Workshop on Robust AI in Financial Services: Data, Fairness, Explainability, Trustworthiness, and Privacy
    [paper]  

  • Visualization of Breast Cancer Metabolism Using Multimodal Nonlinear Optical Microscopy of Cellular Lipids and Redox State
    Jue Hou, Joshua Williams, Elliot L Botvinick, Eric O Potma, Bruce J Tromberg
    Journal of Cancer research, 2018; Vol 78
    [paper]  

  • Characterisation of impaired wound healing in a preclinical model of induced diabetes using wide-field imaging and conventional immunohistochemistry assays
    Mayer Saidian, Jonathan R.T. Lakey, Adrien Ponticorvo, Rebecca Rowland, Melissa Baldado, Joshua Williams, Maaikee Pronda, Michael Alexander, Antonio Flores, Li Shiri, Stellar Zhang, Bernard Choi, Roni Kohen, Bruce J. Tromberg, Anthony J. Durkin
    International Wound Journal, 2018; Vol 16
    [paper]  

  • Feature characterization of scarring and non-scarring types of alopecia by multiphoton microscopy
    Jessica Lin BSc, Inga Saknite PhD, Manuel Valdebran MD, Mihaela Balu PhD, Griffin Lentsch MSc, Joshua N. Williams BSc, Karsten Koenig PhD, Bruce J. Tromberg PhD, Natasha Atanaskova Mesinkovska MD, PhD
    Laserts in Surgery and Medicine, 2018; Vol 51
    [paper]  

  • In vivo multiphoton microscopy imaging of pigmentary skin disorders Mihaela Balu, Griffin Lentsch, Joshua N Williams, Inga Saknite, Karsten Koenig, Anand Ganesan, Bruce J. Tromberg
    Optics in Life Sciences Congress, 2017
    [paper]