AI-Enabled Design Futuring

Last updated May 6, 2024

• Basically like my thesis (See Atlas/Projects/E-MAHINASYON) but with a different methodology
  • Thesis: researches creates AI art
  • CHI Paper: participants create AI art
• Goal: Full paper submission to CHI 25? and/or WIP paper submission to CHI 24
• private/CHIRP 2024

• One Week in the Future: Previs Design Futuring for HCI Research
• Human Enough: A Space for Reconstructions of AI visions in Speculative Climate Futures
• Designing for Narrative Influence:: Speculative Storytelling for Social Good in Times of Public Health and Climate Crises
• Poetics of Future Work: Blending Speculative Design with Artistic Methodology
• Articulating (Uncertain) AI Futures of Artistic Practice: A Speculative Design and Manifesto Sprint Approach

• Workshop where participants create AI art based on the following prompt: “Imagine a day in the life in 2040 for the Filipino”?
  • base idea…you’re essentially exploring the use of AI tools to envision the future…how does the AI influence their perceptions?
• Must educate participants beforehand (e.g. how to write good prompts)
• maybe involve not just have them do AI art, but AI writing as well

• better if participants write the prompts kasi
• kailangan mo igeneralize…don’t evaluate usability of chatgpt….evaluating usecases of AI tools for this context (envisioning the futures)
• Pros and Cons
  • sprint: prepared paper methodology, defined na
  • workshop: faster
  • diary study:
  • ^ kailangan ng justification, prepared
  • ^ write research protocol, expected outputs, present possibilities to Jordan
    • define participants and their background…should they have experience?
    • get at least 5 participants…masyadong marami na ang 10
• Ethical Limitations
  • participant recruitment: consent, recruitment
  • no problem with AI
  • ^ you can make this part of the workshop, their perception on AI…dapat related siya sa main topic
• based on the results of the workshop…this is what we think…and this is how it relates to greater literature
• identify positioning of paper and how it contributes to the greater literature
  • HCI contributions: empirical, artifact
• expected outputs:
  • prompts won’t be very valuable
  • “how do they envision the future”?
  • now that we have AI genertation tools, what future do they see based on the tools?
• another consideration: technical set-up
  • set up discord server for the participants, connected to Richard’s computer, Midjourney
    • just use one discord account, and send multiple prompts (for video)
  • issue: Stable Diffusion is pretty resource intensive
  • Dall-E: not good quality
• “what if we use generative AI tools to enhance the D&D storytelling experience?”
• take exact same 1 week into the future workshop….
  • don’t draw…generate with AI tools
  • if you follow structure, you can ask them to generate multiple images
• do workshop parts in FigJam open session or Miro
• Timelines
  • If you want to aim for CHI 24: late breaking work (like preprints or posters) January 25, 2024
    • pwede siyang remote
    • considered semi-archival, which means you can reuse your paper in the next CHI (if you’re going to make it a full paper)
  • you need to have not just research done, but also have the paper written
    • paper length: 8 pages long, excluding references, double column format
  • You should be done with the workshop by October
    • you can do this within the next 2 months, just figure out logistics
    • Recommendation: Do it ASAP! by middle of first semester
    • email Maam Nikki so that you can get clearance pa
      • CHI paper is small part of thesis (the research part), you have to ask for permission on this
      • the rest of the process is the thesis na your creative work
• make research participants background more specific
  • “They self-identify…”
  • how many years of experience?
  • for students, they should probably come from specific majors
  • pre-filter survey

Title of Project: Keywords: Generative AI, Design Futuring, Speculative Design,

• Rationale
  • Provides the justification or reasoning behind the research project. Explains why the study is necessary and why the research question or problem is worth investigating.
  • Discuss the background context of the research, including any gaps in current knowledge or literature, potential practical implications, and the significance of the research problem.
• Objectives
  • Outline the specific goals or outcomes the researcher aims to achieve through the research project. They define what the study intends to accomplish.
  • Clear, concise statements that describe the expected results or milestones of the research. They may include specific research questions, hypotheses, or tasks.

1. In what ways does AI support the generation of ideas for an utopian future for youth in developing countries?
2. What are utopian and dystopian futures typically imagined by youth in developing countries, and what role does technology play in these?
3. How might design fictions as a method of futuring elicit new ideas that speak to concepts of a utopian reality for youth in developing countries?

Notes:

• maybe replace AI with AI art generators? unless ChatGPT will also be used

• AI Art generators make the process of design futuring easier and more efficient
• Imagined futures are deemed “utopian” or “dystopian” depending on the accessibility and impact of technology in these scenarios
• Design methods as a method of futuring can foster critical thinking and imagination

• No AI tool v.s. with AI tool?
  • best tested out in a group setting, like a workshop. creating design fictions w/ ai v.s. w/o
• w/ design fiction v.s. w/o design fiction
  • pre-study and post-study, measure level of critical thinking and imagination? but how

• RQ 1
  • Independent variable: Use of AI-driven idea generation tools
  • Dependent variable: the amount of creative support a user feels during the design futuring process
• RQ 2
  • Independent variable: Imagined/elicited futures from participants
  • Dependent variable: Role of technology in the scenarios? Or perceptions regarding role of technology?
• RQ 3
  • Independent variable: Use of design fictions as a futuring method
  • Dependent variable: Critical thinking and imagination?
• Others
  • Participant demographics: Collect demographic information about your study participants, such as age, gender, educational background, and experience with technology.
  • Cultural context: Consider the cultural context of participants and how it may influence their responses and perceptions.

to see how were gonna do the studies/workshops.

• Empirical
  • The utility of AI art generators in the design futuring process
  • Findings on how much design fictions can foster critical thinking and imagination?
  • Elicited futures from youth in developing countries
• Artifact
  • Workbook (a novel system)
  • Website gallery of participants’ design fiction outputs (if this counts as novel designs)
• Methodological
  • reevaluating design sprint method of Ivanov et. al (One Week into the Future)
  • introducing a “reflective” component based on method from Harrington & Dilahunt (Eliciting Tech Futures from…)

• Only one participant will be recruited for this pilot study.
• The goal of this pilot study
  • Evaluating the impact of introducing a “reflective” component into Ivanov et. al’s design sprint method
  • Evaluating the results of using AI art generators for design futuring
• The participant will undergo a 5 day sprint wherein they will do the following:
  • Get introduced to design futuring + Stable Diffusion
  • Envision utopian and dystopian futures through co-designed? speculative design fictions
  • Produce concept sketches, storyboards, and comics that examine the implications of these future

• Qualitative instruments:
  • 1:1 Interview
  • Content + thematic analysis
• Quantitative instruments:
  • Creativity Support Index
    • Mixed-Initiative Creativity Support Index: MICSI rates co-creative systems along experiential axes relevant to AI co-creation. We administer MICSI and a short qualitative interview to users who engaged with the Reframer variants on two distinct creative tasks.
  • Can be collected through VideoAsk platform

• Assessing effects and potential benefits will be done in a further and larger user study.

• Design workbook

• Create educational content for VideoAsk

• To read (for background and current approaches):

  • Human-AI Co-creation: Evaluating the Impact of Large-Scale Text-to-Image Generative Models on the Creative Process
  • A Poetics of Potential Form: The Role of Architectural Creative Writing in Augmented Visual Imagination (AVI)
  • Trash to Treasure: Using text-to-image models to inform the design of physical artefacts

• https://www.figma.com/community/file/1074773789352934150/Comics-in-Figma
• https://www.figma.com/community/file/1228326072903106794/AI-Comic-A-Day
• To request:
  • https://dl.acm.org/doi/abs/10.1145/3591196.3596616 (background and current approaches)
  • https://dl.acm.org/doi/abs/10.1145/3581641.3584095 (instrument)
  • https://dl.acm.org/doi/10.1145/3596925 (background and current approaches)
  • https://dl.acm.org/doi/abs/10.1145/3591196.3596819

• Limited thinking = limited innovation
  • Role of ICT Innovation in Perpetuating the Myth of Techno-Solutionism
  • Innovation in HCI- What Can We Learn from Design Thinking?
    • While HCI approaches to design - termed human-centered design by Norman, to the potential confusion of anyone from a design thinking background to whom human-centered design may hold other connotations (e.g. [12]) - are considered well suited for incremental improvements, these are not considered suitable drivers of breakthrough innovation. For the latter, Norman and Verganti argue, new technology or changes in meaning are needed.
  • No More Solutionism۟ or Saviourism۟ in Futuring African HCI: A Manyfesto
    • Paradigms informing design and development projects in the global south are rooted in Western epistemologies that are at best biased and at worst racist
    • “specific design projects perpetuate a particular view of prosperity and progression ۋ albeit on neoliberal political appeals that view Africa (and Africanۑs) as social predicaments to be judged, decided, and confronted”
      • HCI research as problem-solving
    • “The fundamental issue with the interventionist approach to design is that social issues are reduced to objects of social engineering that operate on a culture of dependencies and disparities.”
      • “Such a way of thinking in HCI4D has become hegemonic as it is now framed in the name of doing ‘socially good’ research that stereotypes African conditions as dystopia and Western situations as a utopia [139]
      • The fallacy of good: marginalized populations as design motivation
  • How Methods Make Designers
    • Through their combination of lifestyle and method, Silicon Valley models for tech production such as design thinking, startup incubators, lean management, etc. are spreading across the globe. These paradigms are positioned by product designers, politicians, investors and corporations alike as replicable routes to individual and national empowerment. They are portrayed as universal templates, portable across national borders and applicable to local needs.
    • The last ten years have witnessed a standardization of series of design methods that have come to represent what counts as innovative.
    • Prior research of processes of Western hegemony at the site of technology production and computing.
  • Critique of Design Thinking in Organizations: Strongholds and Shortcomings of the Making Paradigm
    • Prevailing design thinking approaches reinforce rather than transcend the product design mindset.
  • Alternate endings: using fiction to explore design futures
    • It has been noted that HCI researchers do not typically engage in critical evaluation of the potential consequences of their work There seems to be little questioning of the assumption that technology will make our lives more enjoyable, easier, better informed, healthier and more sustainable; or of our role as researchers in that process.
    • The envisioning of HCI researchers is typically simplistic, short term and unconvincing from a sociological or psychological perspective. Indeed, considerable concern has been raised over this lack of consideration for human values in technology design
  • “Design Thinking”: Defending Silicon Valley at the Apex of Global Labor Hierarchies
  • The Limits of HCD
    • The human-centered approach to design is very narrow
    • Focus on individual groups may make an experience worse for others
    • user-centred design (UCD)… “is in fact an incomplete philosophy that lacks a sense of responsibility for concerns other than those of the immediate end user”
      • similar types of UCD and HCD-driven designs have contributed to the serious environmental degradation that we are facing today
    • Realpe-Mu ˜noz et al. raise concerns about HCD and ISO 9241-210’s lack of consideration of user security and privacy.
      • They claim that “there is no process, qualitative and quantitative, that describes how to develop and validate systems taking into account the design requirements and principles (also called heuristics) allowing a good trade-off between security and usability, that is a user-centered design process for usable security and user authentication”
    • Heimgartner questioned the international validity of HCD and its ISO, by explaining that they are too rigid to adapt to local cultural contexts
    • A design process or theory that inflexibly ignores security and culture is not suitable for today’s global challenges.
  • The Limitations of User-and Human-Centered Design in an eHealth Context and How to Move Beyond Them
    • HCD Tends to Lead to Sampling Bias
    • End-User Input Might Be Biased and Limited
    • HCD Tends to Lead to Overreliance on (Fresh) End-User Input
    • End Users Are Only a Subset of the People Who Should Be Heard During eHealth Design
    • Understanding the Added Value of HCD Is Complicated
    • HCD Risks of Supporting the Status Quo
      • responses are likely be limited to end users’ ability to envision new concepts
      • though it is the role of the designer to develop new concepts, when we try to understand people’s needs, this is more easily done in the current context and not in the future context for which the design is to be developed.
      • Consequently, in HCD, we have the tendency to design something new for the current world, rather than designing a new world.
      • In order to move beyond our prejudices and current (working) routines, there are several things we can do, but these require us to change how we do HCD.
        • Designers will need to study and understand the history behind the current situation and the prejudices therein
        • Then, in order to envision a new reality, designers might need to resort to different sources of inspiration, besides end-user input, such as feminist and queer theory, art, or the maker culture.
        • Designing against the status quo might mean designing for the long term.
    • Traditional HCD and Designing for Behavior Change Are Not a Good Match
    • HCD Tends to Miss Out on Ethical, Societal, and Political Aspects
      • HCD activities focus on individual users, their context, and their needs and expectations in relation to specific tasks and goals. Thus, HCD tends to prioritize the microlevel rather than the mesolevel and macrolevel.
      • Discrimination may stay hidden if we focus on specific user groups and how technology can support their tasks and goals (eg, supporting a judge in sentencing decisions).
    • HCD Thinks About the Beginning but Not the End
  • Critically Conscious Computing
    • CS teaching, therefore, naturally gravitates toward utopian narratives.
    • Most arguments for teaching CS stem from goals of serving industry or providing literacy for an informed citizenry
    • Predominant CS pedagogy focuses on CS skill development
    • Inclusive design methods also focus on helping designers see past their assumptions. For example, one method called assumption ideation involves analyzing how software excludes and does harm.
    • higher education curricula focus on abstract conceptions of computing:
  • Design justice
    • Call for healthy skepticism of universalist and solutionist notions of design as a way out of structural inequality
    • educator  Sherri  Spelic  writes:  “Design  Thinking  aligns  well  with  a  certain  kind  of  neoliberal  enthusiasm for entrepreneurship and start- up culture. I question how well  it  lends  itself  to  addressing  social  dilemmas  fueled  by  historic  inequality and stratification.”
    • universalist design principles and practices erase certain groups of people, specifically those who are intersectionally disadvantaged or multiply burdened under white supremacist heteropatriarchy, capitalism, and settler colonialism.
    • Situated > universalist knowledge
      • No one group has a clear angle of vision. No one group possesses the theory or methodology that allows it to discover the absolute ‘truth’ or, worse yet, proclaim its theories and methodolo-gies as the universal norm evaluating other groups’ experiences.
    • Much, or perhaps most, design work imagines itself to be universal: designers intend to create objects, places, or systems that can be used by anybody. Design justice challenges the underlying assumption that it is possible to design for all people
• Researchers in developing countries
  • Theory, Practice and Policy: An Inquiry into the Uptake of HCI Practices in the Software Industry of a Developing Country
    • Focus of curriculum: usability; interdisciplinary education is discouraged
    • Tacit knowledge: Practitioners are not able to express what they do.
  • Listening to the voices of early-career researchers in the Global South so that we can better support them to thrive
    • Our ambitions for research and knowledge systems rest significantly on the emerging and next generation of researchers. These are the individuals who will investigate new technologies…
    • Early career researchers in low- and middle-income countries face many challenges, but they are positive about their careers – 70% are satisfied with their job and almost two thirds have a good work/life balance. But 90% feel that they need more training and support to progress, and less than half feel secure in their jobs.
    • Researchers are eager to contribute to scientific knowledge and to make a difference to society by solving real-world development problems, but most are assessed by the number of papers they produce, or the journals they publish in, not by wider measures of impact
      • “I think it’s reinforcing what some of us have been saying for a long time, that actually the usefulness and the use of research is what’s really important. Whereas in terms of research excellence, for so long, it’s been driven entirely by the kind of academic quality and rigour of the research and the originality of the research question. I think this is a really strong statement from researchers in the Global South, that it’s really important that the research which is done is done to address a problem and finding solutions to a problem.”

• Design Heuristics for Artificial Intelligence: Inspirational Design Stimuli for Supporting UX Designers in Generating AI-Powered Ideas
  • Design heuristics themes
    • Decision-making
    • Personalization
    • Productivity
    • Security
• AI for the Generation and Testing of Ideas
  • generative AI can increase idea creation by removing human ego from the discussion.
• The effect of AI-based inspiration on human design ideation
  • we examined how AI inspiration based on conceptual similarity influences distinct properties of idea generation in the design process.
  • In response to the research question, the participants produced more novel ideas (novelty), new ideas (variety), and ideas (quantity) in the treatment condition than in the control condition and the increased novelty, variety, and quantity influenced the temporal pattern of the ideation process.
  • The interpretation of our results suggests that AI-based conceptually similar inspirations (semantically close inspirations) can increase the number of new and novel ideas when engaged in a design task.
• A framework of artificial intelligence augmented design support
  • Gaps in AI-augmented design
    • Firstly, existing design support overlooks some critical design stages. Design ideation (referred to as “conceptual design” in design research) involves creative and reflective thinking. It is challenging to provide external support for this high-order thinking.
    • Secondly, design ideation involves complicated cognitive processes and has a broad ontology context. Significant design factors need to be considered in the process of designers developing parallel and nontrivial paths to a problem, however, they have not been well examined in design situations.
    • Background and progress
      • the potential scope of “intelligent” design support has not been examined.
    • In-improving design problem
      • existing computational tools overlooked the pivotal framing process (“design as reflective process”) that yields tacit design knowledge for such distinct and implicit situations
      • In design practice, computational support shows limited success in early design activities (Bernal et al., 2015; Gero & Kannengiesser, 2004) due to “invariants” of design problem space.
        • tacit design knowledge is usually generated
      • “Invariants” refer to features of a structured information-processing model that do not all occur in problem space
      • Design problem spaces exhibit major invariants across design problem-solving situations and major variants across non-design problem-solving situations in which computational models can fail.
        • E.G. a design-agent only works if the conditions of a well-defined design task are specified and developed, i.e. in the late design process where concrete forms and solutions are being generated.
        • Generative design tools are limited to the target variables defined by the desigenr
      • insufficient HCI examination for problem (re)framing processes
    • Structured design process model
      • In-improving design problem determines that the design process is intrinsically iterative, parallel, and strategically complicated
      • Key research problem in AI-based design: design process modelling
      • Assumption of structured design process fails to consider invariants….creative processes happens simultaneously
  • A framework for design ideation with AI
    • Built on the following frameworks
      • Design optimization framework: represents fundamental processes of designing in a dynamic world.
      • Generic-design hypothesis: characterizes complex design “situations” on three main dimensions, the design process, the designers, and the artifact
    • How AI supports human design ideation
      • Discovery: AI’s computational mechanism is triggered by designer’s selection and parse initial information (e.g., design research notes, interview transcripts, inspirational images) into insightful representations (e.g., entity patterns, logical representations).
      • Reframing: suggestions and references are produced to juxtapose the information from the discovery phase
        • E.G. informing potential design states by AI’s predictions.
      • 3 roles of AI
        • AI as Representation creation: providing inspirations by suggesting texts or images
        • AI as Empathy trigger: supporting the designer’s descriptive thinking
        • AI as Engagement: helping the designer avoid fossilization and perform typical design actions
  • Design principles with AI
    • Start from scratch
    • Knowledge-driven
    • Decomposition and integration
  • Challenges
    • Visible AI for trust and learnability
    • Controllable AI for interoperability and reliability
    • Adaptive AI for variant backgrounds, attitudes, and expectations
• Too Late to be Creative? AI-Empowered Tools in Creative Processes
  • Stages of the Creative Process
    • Q&A Stage
      • While individual scholars have ofered their unique and/or domain-specifc proposals, they commonly suggest that a creative process typically embarks on understanding the creative problem itself.
      • During this initial stage, creators would gather relevant information (e.g., secondary research, observation, survey) and perform goal-setting, while the order of these actions may difer case-by-case.
    • Wandering Stage:
      • This is when creators start to initiate and play around with some scattered, premature pieces of thoughts —they may not be complete enough to directly address a creative problem, but they serve as the nutrients for more formal ideas later on.
      • During this stage, even though a person is not consciously producing workable ideas, their brain continues to search for concepts and opportunities that can fuel the formation of creative strategies.
      • Specifcally, researchers refer to this experience as incubation and emphasize the power of “letting an idea sit”
    • Hands-On Stage:
      • when creators actually start to work on solving a problem. This process is often initiated by generating a large number of possible ideas (i.e., solutions to the creative problem)
      • Several design disciplines refer to this process as brainstorming and utilize a variety of techniques to drive such activities
      • With an initial set of possible solutions, creators may evaluate (screening out the good from the best), combine multiple compatible ideas, and select a small set of “candidates” to work on further revision and improvement
    • Camera-Ready Stage:
      • In the fnal step of the creative pro-cess, creators fnalize and execute ideas into presentable, “client-facing” formats, allowing them to “sell” their ideas to their intended audience.
      • During this stage, creators leverage professional skill sets, tactics, and tools to externalize intangible concepts into concrete forms.
  • HAII challenges
    • Divergent Challenges
      • when users apply the technology to perform generative tasks, where AI involves in creating and/or delivering certain products
    • Convergent challenges
      • how AI guides users to make decisions, such as selecting the best option out of numerous alternatives or identifying trends from massive data.
    • Collaborative challenges
  • Categories of AI co-creative tools
    • The Editors: facilitate various execution processes, allowing users to carry out content editing at ease.
    • The Transformers: alter and convert content from one form to another
    • The Blenders: combine two or more creative elements to breed new ideas and outputs.
    • The Generators: produce ready-to-use creative outputs based on guidance and/or constraints inserted by users.
  • SWOT
    • Strength: Strategizing Human-AI (Non-)Creative Task Assignment
    • Weakness: Shying away from HAII and Workplace Reality
      • Due to a lack of insights into how the tools work, what level of complexity to expect in the output, and how machines optimize their behaviors, users have little choice but to be more hands-of during the interaction experience.
      • This particularly occurs when users are interacting with the Generators. Moreover, when generative outcomes turn out to be unsatisfying, there is often limited guidance for how improvement can be made systematically
    • Opportunity: Breaking Misconception & Going beyond the Creativity Domain
      • misconceptions
        • scholars commonly suggest though machines can produce artistic outputs, they lack the ability to evaluate and distinguish the mundane from the groundbreaking masterpiece
        • rise of concerns for accountability, responsibility, and fairness in HAII have also put a pause on whether AI should participate in goal-setting
      • do “non-human” ways of AI reasoning and behaving remain inappropriate under the context of seeking creativity
      • recent work has shown that, even without AI systems understanding and seeking creativity as the target for optimization, their very distinct approaches to “thinking” can introduce intriguing perspectives that may fall out of humans’ sights
      • I see the data-driven, non-human mind of AI ofering much potential in re-defning and re-positioning the early stages of the creative process. Similarly, when it comes to evaluating creative work, AI may not “comprehend” nor base on the meaning of creativity to select great content, but it can identify outstanding pieces that may not otherwise be noticed by humans
    • Threat: Guarding the Territory of Human Creativity
  • Conclusion
    • future AI co-creative tools should further explore the possibilities to offer support at various steps (especially the early stages) of the creative process, since producing creative work requires more than generating and executing ideas.
    • encourage designers and developers of these tools to address common HAII challenges, informing how users can interact, intervene, and determine roles in human- machine partnerships.
• A literature review on individual creativity support systems
• How to use Generative AI as a design material for future human-computer (co-)creation?
  • Among the three papers related to speculative and critical design, only [ 27 ] specifically focuses on computational co-creativity. This seems to indicate that these approaches are underrepresented in the research field.
  • Speculative and critical design on computational creativity, however, is underrepresented
  • this study put forward the urgent need to speculate on the potential of this incredibly fast-evolving GenAI technology, thinking about the future user experiences and interaction architectures of GenAI applications.
  • We propose that research through speculative and critical design (e.g., co-speculation workshops with focus groups) would be a way to help understand how to develop GenAI systems in the next 5 to 10 years
  • How to speculate on GenAI
    • Research Through Speculative and Critical Design
      • Cards/Research through Design: actual artifacts are designed and made in order to respond to specific research questions
      • Cards/Speculative design: aims at imagining alternative futures and how the designed objects would alter, shape, and redefine our human world.
      • Cards/Critical design: aims to challenge our assumptions of how these designed objects would fit in our human world
      • Thus, research through speculative and critical design allows us to imagine different futures in concrete ways, which helps us to prepare for them.
        • Purpose of this project’s artifact? for target audience
    • Co-speculation Workshops
      • To place end-users of design in focus, we plan to carry out co-speculation workshops with interviews or focus groups.
      • Cards/Co-speculation is a collaborative method within speculative design practices that incorporates non-design experts
    • Co-speculating with sketches and prototypes
      • We plan to use design sketches, user experience scenarios, and low-fidelity prototypes as conversation prompts in a series of co-speculative workshops
      • Sketching is a fundamental part of the design process that helps designers generate and discuss design ideas

• Prompt Engineering in Medical Education
  • In medical education, prompt engineering can create realistic patient scenarios, generate multiple-choice questions, or provide explanations of complex medical concepts. Prompt engineering can also control the model’s output’s length, complexity, and style. For example, prompts can be designed to elicit short, simple responses for beginner students or more complex, detailed responses for advanced learners. Prompt engineering can also generate messages appropriate for patient education and mass media campaigns. Moreover, prompt engineering can help minimize potential pitfalls, such as the generation of incorrect or misleading information. Educators can guide the model with carefully crafted prompts to provide more accurate and reliable information.
  • Prompt Engineering for Large Language Models to Support K-8 Computer Science Teachers in Creating Culturally Responsive Projects
  • Prompt Engineering For Students of Medicine and Their Teachers

• One Week in the Future: Previs Design Futuring for HCI Research
  • Sprint participants prepared document summarizing process, reflections, and outcomes
    • Researchers then examined these summaries and extracted a collection of unique observations, which were then clustered into themes
  • Futuring opportunities
    • Alternate mediums and concept fidelities
      • Sprint and workbook format can support alternatives
    • Rapid scene building tools
      • Plethora of available assets is needed to quickly layout a a 3D scene; without such resources, it would be too timely/costly for an individual designer
      • But relying on preexisting assets also constrained the design of concepts
• FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design
  • We developed AI models by externalizing three cognitive operations (extending, constraining, and blending) that are associated with divergent and convergent thinking.
• Developing an AI-based automated fashion design system: reflecting the work process of fashion designers
• StoryDrawer: A Child–AI Collaborative Drawing System to Support Children’s Creative Visual Storytelling
  • The system includes a context-based voice agent and two AI-driven collaborative strategies: the real-time transformation of children’s telling into drawings, and the generation of abstract sketches with semantic similarity to existing story content.
• Recipe 2.0: Information Presentation for AI-Supported Culinary Idea Generation
• Using Artificial Intelligence to Generate Master-Quality Architectural Designs from Text Descriptions
  • To help ordinary designers improve the design quality, we propose a new artificial intelligence (AI) method for generative architectural design, which generates designs with specified styles and master architect quality through a diffusion model based on textual prompts of the design requirements.
  • Compared to conventional methods dependent on heavy intellectual labor for innovative design and drawing, the proposed method substantially enhances the creativity and efficiency of the design process. It overcomes the problem of specified style difficulties in generating high-quality designs in traditional diffusion models.
• Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools
• Generative AI for Concept Creation in Footwear Design
• Collaborative Diffusion: Boosting Designerly Co-Creation with Generative AI
• Closer Worlds: Using Generative AI to Facilitate Intimate Conversations
• When happy accidents spark creativity: Bringing collaborative speculation to life with generative AI
  • Introduction
    • Participants collaboratively speculated on utopian ideas for the future. These speculations were then fed as text prompts into a generative AI model to visually manifest them. We conducted a series of user interviews to learn about the experience from participants, and surface key themes.
    • In this paper, we present a field report of the experience and use the system to trace broader questions about the social and collaborative aspects of creativity, such as when generative visual imagery can inspire ideas about the future, and how the variety/fidelity trade-off in generative models might impact creativity
    • Our work builds on previous work (Rafner et al., 2021, 2020) which builds tools for people to express both hope and anx- iety for the future via blending with generative models, and investigates image features associated with dystopian versus utopian visions of the future
  • Dangers
    • AI-generated images are bounded by training data, which inherits historical biases and cultural practices
    • Relatedly, if such an approach becomes commonplace, there is the risk that such visualization strategies could be a crutch if used too much, with users becoming overly reliant on a machine’s vision of the future rather than their own.
    • Finally, there is the risk that anthropomorphizing the AI can undermine human credit and responsibility
  • Future Work
    • Future work could explore how this approach works for other communities with distinct cultures and practices, and howthe effectiveness of the approach varies across community contexts
    • Future work could explore digital versions of these methods, such as an online interface for collaborative prompt generation, an online gallery for the prompts and their cor-responding images, or real-time dialogue with a generative model.
    • Future work should consider more explicitly users’ understanding of the involved technology, as well as consider other types of models, such as physically-informed models of climate futures
    • Finally, there is the possibility of using this method in diverse settings.
• Utopian or Dystopian?: using a ML-assisted image generation game to empower the general public to envision the future
  • The current study affirms the potential of human-GAN interactions with a suitably designed interface to afford the expression of recognizable ideas, thoughts, or concepts.
  • Future work with a larger number of raters could potentially examine systematic differences among individual images or creators at a finer grain.
  • Further, we expect there to be national and cultural differences in dystopian and utopian aesthetics
    • WEIRD vs non-WEIRD researchers
  • A tool like this will make it possible to study how people think about creative expression, and provide data for studying at scale and in depth what kinds of features of images people attend to when creating or interpreting images, and how these features connect to users’ underlying concepts.
  • These high ambitions can only be achieved with a detailed understanding of both the optimal technological support as well as the thought processes in both creators and raters and our quantitative analysis is therefore a crucial first step in this direction.
• Custom GPTs focused on generating future scenarios
  • Includes DALL-E
  • Examples: Alternate Reality Explorer; Future Explorer

# **Explaining Cards/Design Futuring

• Expanding Modes of Reflection in Design Futuring
  • Design not used to solve an immediate problem, rather to produce knowledge through discourse
  • Concerned with radical future alternatives
• Calling for a Plurality of Perspectives on Design Futuring
  • We use design futuring as an umbrella term to loosely refer to such approaches (e.g. speculative design, discursive design, design fiction), which are “concerned with future alternatives” and which seek to “produce knowledge through debate, contestation, refection etc.”
• Speculative Design in HCI: From Corporate Imaginations to Critical Orientations
  • history of how speculative design was introduced to HCI publications through corporate design research initiatives from the RED group at Xerox PARC.
  • Our argument is that third wave, critically oriented, speculative design “works” in HCI because it is highly compatible with other forms of conventional corporate speculation (e.g. concept videos and scenario planning).
• “All that You Touch, You Change”: Expanding the Canon of Speculative Design Towards Black Futuring
  • One of the more imaginative aspects of interaction design and HCI, design futuring - the concept of reimagining both desirable and undesirable aspects of our world and technology’s role in it - is one of the more radical approaches for how we might conceptualize new trajectories of our world
  • science fiction has laid the blueprint for innovation in HCI
    • absence of intersectional identities poses the question of whose voice is considered in envisioning that future
  • design futuring becomes a pathway for envisioning what could be, a departure from present day circumstances with a license to imagine infinite possibilities
  • A small number of scholars in the HCI and interaction design space have begun exploring these frameworks as approaches to shift whose voices are amplified in design futuring as well as how methods reflect these lived experiences. Much of this research has used speculative fiction to elicit the ways descendants of the Black diaspora activate imaginations which shifts the vision of the future to be centered on the needs and desires of Black people.
  • expanding our design canon should consider two major areas of work that engage Black futures: The first being work that pushes for equitable societal futures for Black populations where technology may or may not play a role, and the second being work that engages Black folks in architecting the future of technology itself.
    • From an Afrofuturism perspective, technology is a means to an end where Black people envision a future where racism and other forms of oppression do not exist and we as human beings surpass our frailties and become our best selves. In this vision, technology may help to create an imagined future where Black people live in harmony with all people.
    • In this second area of considering Black futures, Afrofuturism positions Black people as innovators, inventors, and producers of technology used to create more equitable futures that benefit everyone.
• Ancestral and Cultural Futuring: Speculative Design in an Indigenous ovaHimba context
• How HCI Integrates Speculative Thinking to Envision Futures
  • Many researchers have pointed out the frequent lack of clarity about how Design Futures will generate new knowledge and contribute to HCI. This includes the presentation and experience of speculative work for participants and how user engagement will be understood. The lack of explicit guidance in HCI for anticipating multiple futures and assessing the long-term effects of technologies highlights the importance of interdisciplinary collaboration and the use of futurological methods.
• Design fictions for learning: A method for supporting students in reflecting on technology in Human-Computer Interaction courses
  • Design fictions could be helpful in the context of academic education to teach fundamentals of technology design/HCI, being capable, at the same time, of conveying a critical perspective on the discipline, helping students reflect on the assumptions and ramifications of designing technology.
  • we aim to use design fictions as a “tool for reflection” in order to support students in thinking of emerging technologies: design fictions may help students look beyond the short-term implications of designing technology, encouraging them to explore its systemic consequences, critical issues, and hidden presuppositions.
  • Such a perspective somehow represents a reaction against the tendency of design to imagine near and utility-driven futures, without exploring the ambiguous long-term implications of technology
  • On the one hand, as Linehan et al. (2014) highlighted, the envisioning of HCI scholars has been recognized as often simplistic and short term; on the other hand, in design practice there seems to be little questioning of the assumption that technology will transform our lives into something better.
• Design Futuring for Love, Friendship, and Kinships: Five Perspectives on Intimacy
  • Design futuring and related methods ofer approaches for imagining alternative future technologies, practises, and systems
  • Design futuring is a term we use to refer to an orientation to design that among other approaches encompasses Speculative Design and Design Fiction. It involves the creation of props, either artefacts or stories, that explore how technologies might transform the social practises and contexts in which they are embedded
  • Within HCI, design futuring has already expanded into everyday contexts…
• Speculative Blackness: Considering Afrofuturism in the Creation of Inclusive Speculative Design Probes
  • Methods of speculative participatory design are useful in imagining futures among marginalized groups while negotiating existing societal constructs.
  • Scholars such as Winchester and Baraka provide a foundational framing of the Black ethos as a design lens for more inclusivity in technology innovation
  • “This narrow perspective indicative of the status quo not only constrains design exploration but also fosters future technological solutions that are ignorant of the needs and considerations of often marginalized and disenfranchised groups, such as Blacks in the U.S.
  • Afrofuturist speculative design recenters narratives about who is included in the future, who shapes it, and what is worthy of speculation
• Use cases of AI in futuring
  • https://www.timesnownews.com/viral/ai-generated-images-delhi-air-pollution-future-by-madhav-kohli-scary-see-viral-twitter-pictures-article-96982637
  • https://edition.cnn.com/style/article/ai-architecture-manas-bhatia/index.html
  • https://www.marketing-beat.co.uk/2023/04/18/earth-day-wwf-campaign/
  • https://sdw.designsingapore.org/events/education-reimagined-designing-scenarios-for-the-future-of-learning-with-critical-design-thinking-and-generative-ai/
  • https://www.geekwire.com/2023/ai-envisioned-the-future-of-downtown-seattle-heres-where-it-fell-short/
  • Delphi-based visual scenarios: An innovative use of generative adversarial networks
  • Explore the Future Earth with Wander 2.0: AI Chatbot Driven By Knowledge-base Story Generation and Text-to-image Model
    • Through journeys with visitors from all over the world, Wander demonstrates how AI can serve as a subjective interface linking fiction and reality
  • HUMAN ENOUGH: A Space for Reconstructions of AI visions in Speculative Climate Futures
  • Design Futures with GAI: Exploring the Potential of Generative AI Tools in Collaborative Speculation
    • Research questions
      • How might AI-based tools assist in scanning signals
      • How might AI-based tools inspire students to imagine the future scenario?
      • What support do students need to promote the design concept that addresses future challenges?
    • Methodology
      • collaborative workshop with GAI-embedded workbook
      • based on Double Diamond Design Model
      • Participants are already trained in GAI tools (e.g. ChatGPT, StableDiffusion, Midjourney)
      • Session Structure
        • 1. Discover the trends and changes:
          • STEEP framework
          • Here, ChatGPT plays the role of various experts in five domains to gather information that may influence the future
          • ^ we would need a good prompt for this
        • 2. Define future challenges
          • List potential future scenarios + challenges
          • ChatGPT can also help with this
        • 3. Develop design ideas to respond to these futures
          • come up with artifacts based on scenarios
          • describe the impact of these designs
        • 4. Deliver the speculative objects for discussion
          • describe future target audience
          • design final artifact
          • storyboarding
      • Data analysis
        • Storyboards, design scenarios, and design works were all analyzed through thematic analysis.
        • In order to further understand the experience of participants co-designing with various GAI tools, five Likert scale was adopted to investigate, involving the effect of GPT expert signal scanning, quality of GPT assisted storytelling, method, and effect of using GAI tools to generate images.
        • During several stages of signal scanning, scene structure, and product design, the perception of the effectiveness of AI assistance was descriptively statistically analyzed. In addition, the researchers conducted group interviews with each group on days 2 and 4 of the workshop, with the main purpose of analyzing the potential and limitations of human-computer interaction
• Datasets
  • https://forum.effectivealtruism.org/posts/McMwdgLZnuiCdsrrb/a-dataset-for-ai-superintelligence-stories-and-other-media
  • https://airtable.com/appYdspFFPWGtSBLA/shr5EIpLNHB7o2q9Z/tblMVjRvMKVNkoZVg?backgroundColor=cyan&viewControls=on

• Perceived problem: limited innovation
  • Emerging researchers will be responsible for future innovations in the Global South…however, many of them struggle in designing and undertaking research due to a lack of funding, support, visibility and recognition
    • insert statistic here comparing researchers from developing vs developed countries?
  • Many of these researchers are motivated by the desire to contribute to scientific development and to wider society emerged very strongly. They also wanted to be novel or innovative.
    • However, most are assessed by the number of papers they produce, or the journals they publish in, not by wider measures of impact.
  • Proposition: We also believe that researchers’ innovation are also constrained by limited methods, at least in the field of HCI
• Root cause: Limited thinking
  • the vision of the future described by HCI researchers and practitioners is typically utility-driven and focuses on the short term. It rarely acknowledges the potentially complex social and psychological long-term consequences of the technology artefacts produced. Thus, it has the potential to unintentionally cause real harm
  • Silicon Valley’s methodological hegemony
  • Transition: need for more radical methods
• Introduce design futuring and its benefit for research
• This project’s contributions
  • builds on one week into the future sprint by doing an AI-supported sprint
  • researches impact on ECR from developing countries?

• Section 1: Design futuring in HCI
  • Talk about the use of design futuring in HCI?
  • Mention examples
    • The motivation for these studies is to make the world a better place for everyone by designing inclusive and fair (future) technologies
  • How design futuring benefits minority groups
    • Black futuring frameweworks (like Afrofuturism) used as design lenses
• Section 2: Generative AI and human creativity
  • AI has been used to help with human creativity… like ideation. (general)
  • Research gap: speculative and critical design is an underrepresented approach in computational co-creativity
  • Keywords: AI-supported idea generation, creativity support tools/systems
  • Examples: fashion design, visual storytelling, recipe making
  • Mention of examples of people using AI to not just imagine new creations (e.g. architecture), but also images of the future
  • Relation to this project: it could help with design futuring?
    • there are already existing GPTs

• Part 1 - Pilot study: One-shot prompt engineering workshop with 5 researchers-in-training
  • One-Shot prompting: refers to a strategy in which the model is shown one task-specific example before presenting the actual prompt.
    • What is the sample? The research review canvas?
    • One-shot prompting can be combined with other natural language processing techniques like dialogue management and context modeling to create more sophisticated and effective text generation systems.
  • Here, we’d find the best prompt for being exposed to multiple images of the future
  • Output: description of scenario + accompanying image
  • This would then inform the development of dataset + model for future artifact: AI-enabled version of design futuring sprint workbook? (like a custom GPT)
• Part 2 - Full Design sprint
  • Possible conditions for study:
    • testing the impact of the AI-enabled CST
      • with the generative AI system
      • just using the design sprint workbook???
    • comparing researchers from different demographic
      • researchers-in-training in WEIRD countries v.s. developing countries
• Part 3 - Follow-up discourse (group discussion based perception survey)
  • Researchers’ outputs are then compiled in a website, which another group of participants then evaluates
• Data collection and analysis:
  • pre and post-interviews or FGD with the researchers?
    • Post-sprint
      • Reactions to AI-provided concept
      • Estimated year
      • Estimated sentiment
      • role of self
      • likelihood of occurence
      • desired modifications to concept
  • thematic analysis
    • applies not just to interviews…but maybe also to the outputs described?

The future of technological innovation in the Global South rests partly on the next generation of researchers. These early career researchers (ECRs) are motivated by the possibility of contributing to scientific development and to wider society, along with the desire to be novel or innovative. Unfortunately, many of them struggle in designing and undertaking this kind of impactful research due to a lack of funding, support, visibility and recognition. Additionally, in the human-computer interaction (HCI) field, ECRs could also be blocked by the limitations of established methods.

One such example is human-centered design (HCD): “an approach to interactive systems development that aims to make systems usable and useful by focusing on the users, their needs and requirements, and by applying human factors/ergonomics, and usability knowledge and techniques”. Its widespread influence can be found in its core adoption across many computing, design, and management projects in industry, academia, and government. But while it is effective at delivering popular and effective interactive systems, HCD’s focus on individual people or groups gives it a narrow temporal and contextual focus.

One limitation of HCD is its limited perception of time, which can be seen in designers’ tendency to design something new for the current world, instead of designing a completely new world. This is because HCI design and development is an iterative engineering process, wherein design is gradually improved by a growing understanding of the users and the context of use. Thus, while HCD may be suitable for incremental improvements, it is a not a fit driver of breakthrough innovation. Another flaw of HCD is that it often misses out on ethical, societal, and political aspects. HCD and its ISO, ISO 9241-210, have been criticized for lacking consideration of user security and privacy, along with being too rigid to adapt to local cultural contexts. And since human values are often not touched on in HCD, HCI researchers usually do not critically evaluate the potential consequences of their work.

If ECRs in the Global South believe that high-quality research must be beneficial and groundbreaking, then it is crucial for them to utilize more radical design methods. Enter design futuring: a type of approach to design that is “concerned with future alternatives” and seeks to “produce knowledge through [discourse]” (instead of being used to solve an immediate problem). It not only helps design researchers generate and communicate new ideas, but also contemplate the social and technical implications of their work. However, producing design fictions often requires proficiency in creative software, preexisting assets, and time — things many researchers may lack, especially if they’re located in the Global South. Thus, there is a need for the development of creative tools that can support rapid prototyping for design futuring.

Given its meteoric development, generative AI (i.e. AI technologies that automatically generate written or visual content based on text prompts) has the potential to bring these kind of tools to life. In line with this, we propose an AI-enabled creativity support tool that rapidly envisions future aspects of a researcher’s area of interest, building on Ivanov et. al’s design futuring sprint method. This paper is a preliminary study that explores how generative AI can help ECRs in the Global South imagine, evaluate, and consider the implications of their works, so that they can work towards high-quality innovation. This study aims to make a dataset of prompts for fine-tuning a model designed to generate fictional depictions of the future; this would then be used in the development of the proposed creativity support tool.

• Why is there a need for ECRs in the Global South better methods?
  • universalist and solutionist notions of design as a way out of structural inequality
  • Western hegemony is problematic
    • such templates reflect universalism and solutionism
  • Paradigms informing design and development projects in the global south are rooted in Western epistemologies that are at best biased and at worst racist

Background: Design Futuring in HCI Research + Practice

Related Work: AI-Enabled Creative Tools for Design Futuring Generative AI tools have made creative exploration more accessible, thanks to rapid advancements in large language models (LLMs) and diffusion models. Popular examples of these tools, such as ChatGPT and Dall-E, implement a low-floor, high-ceiling and wide-walls principle: they have a low barrier of entry (low floor), allowing users to achieve high-quality outputs (high-ceiling) that can extend to extend to a broad range of applications (wide walls). Acting as an “engine for the imagination”, generative AI has been specifically been used to support the creative processes of ideation and creation. While it has already been applied to creative activities like visual storytelling and culinary innovation, it is most especially prominent in design, with fields like industrial design, fashion design, and architecture.

Despite generative AI’s widespread adoption in several creative activities, its use in speculative and critical design remains underexplored. But generative AI has already shown to be useful across a spectrum of envisioning speculative futures, from practical forecasting to creative storytelling. Applications of generative AI have started to demonstrate their potential for collective design futuring, encouraging discourse on societal issues like climate change. Overall, all of these examples show that “AI can serve as a subjective interface linking fiction and reality”. Our project contributes to this growing body of work by exploring how generative AI can be act as an interface that connects the realities of researchers with the futures they desire.

There is an opportunity to assess how generative AI can play a more active role — what if it was not just a tool, but also a collaborator? AI-driven co-creative tools are less involved in the early stages of co-creation (i.e. researching, incubating). This is because of the misconception that machines lack the ability to evaluate and distinguish mundane outputs from masterpieces, along with concerns for accountability, responsibility, and fairness in human-AI interaction. However, the data-driven, non-human mind of AI offers much potential for redefining and repositioning early stages of the creative process; even without setting creativity as the target for optimization, AI can introduce intriguing perspectives that may fall out of humans’ sights. For example: when Lyu, Hao, and Yi embedded generative AI into the design futuring process, it served multiple functions: signal scanning, scenario development, world-building, and more.

But AI must still be used with caution; without intentional human guidance, machine-generated visions may be harmful and insubstantial. For instance, in response to Seattle leaders using AI imagery to promote a “downtown of the future”, Wolfe discussed the following challenges that AI-enabled design futuring can bring about: (1) the data being used to train AI may contain biases and inequalities, which may lead AI to perpetuate marginalization; (2) AI may be unable to account for evolving needs, given that it is trained on a limited set of data; (3) without definitional prompts, AI lacks contextual understanding, which can lead to generic outputs; and (4) AI may prioritize standards like efficiency and aesthetics over the human experience. To ensure human-centered outputs, an AI-enabled design futuring tool will require meticulous prompt engineering. It will also require user education. Usually, users are forced to be hands-off when using AI co-creative tools due to a lack of knowledge on how the tools work, what level of complexity to expect in the output, and how machines optimize their behaviors. This particularly occurs with generative AI; when users get unsatisfying outputs, they have limited guidance for making improvement. In light of these considerations, this study would like to investigate how to develop a thoughtful AI futurist.

• Background: Design Futuring in HCI Research
  • Design futuring for researchers
    • One Week Into the Future
  • Design futuring for marginalized communities
    • Black frameworks for Black communities
  • Research gap: we can see how futuring can help HCI researchers and marginalized communities, but we don’t know how it could benefit researchers from marginalized communities

We will be conducting a pilot study primarily aimed at answering our research question: How can generative AI best support the design futuring process?

Participants Through convenience sampling, we will be recruiting five participants: computer science students working at a university research laboratory in the Philippines. Given their technical background, they are expected to already be familiar with using generative AI tools (e.g. ChatGPT, Dall-E). They should also be currently working on a research project, since they will be required to explore it in the pilot study.

Study Design The participants will go through a 90-120 minute workshop based on the first day of Ivanov et. al’s design futuring sprint. We will be using a within-subjects design where all participants do the activities on their own, and then try doing the same activities with the help of generative AI. The workshop will consist of the following parts:

• Introduction: The workshop facilitator will give a participants an overview of the workshop. Then, the participants will be shown guidelines on prompt engineering, to help them with the upcoming workshop activities.
• Research Review: The participants will write about their area of study, using the canvas template from Ivanov et. al’s workbook (Figure 1). Then, after writing their own answers, the participants will use ChatGPT 4 to answer each section in the canvas. Answers written by both the participants and ChatGPT will be placed beside each other for easy comparison.
• Generating Exercise: Once they have an overview of their domain, the participants will use DALL-E 3 to generate fictional depictions of the future, in response to six prompts (Figure 2). After generating images on their own, the participants will refer ChatGPT 4 as an industry expert, asking it how it would answer those prompts. These prompts would then be translated into images with ChatGPT 4. Similar to the previous activity, images made by descriptions from both the participants and ChatGPT will be placed beside each other for easy comparison.
• Focus Group Discussion: After a short break, the facilitator will interview the participants about their experience with design futuring, and how generative AI affected this.

Documentation and Analysis The workshop will take place remotely via Zoom, allowing for easy recording. We will be using a shared Figjam whiteboard throughout the whole workshop; all prompts and outputs (i.e. answers, images) will be placed here. To evaluate the participants’ experience with generative AI, we will be transcribing and conducting a thematic analysis of the FGD.

• https://ai.torchbox.com/thinking/2023-06-16-generative-ai-powerful-tools-to-imagine-explore-better-futures

• explore how AI-enabled design futuring works for different groups. how national and cultural demographics (e.g. WEIRD) influence aesthetics. what do people consider positive and negative?
  • can tie in how to make AI adaptable? and less biased?