This class is for anyone trying to use code to tell a story, compose music, or embue a physical object with the power of emotional expression. We will be looking at examples from a broad array of media and disciplines from the column structure of Greek temples to music (classical and contemporary) to poetry to paintings to story-driven video games and yes Youtube "cat" videos.
You will be working from code examples and simple interfaces I will create to help you play with different computational techniques. Our aim isn't to invent fancy new algorithms. Instead we're using computation and programming as a tool to help us decode the mechanisms that drive compelling emotional arcs. More details to come! Full description.
Have questions? Email me at [email protected] | Sign up for Office Hours | Join our class mailing list | Homework Wiki
###Week 1: Seeing, Hearing, Sensing, Feeling Through the Lens of Patterns
- What is narrative arc? What is composition? What are its basic building blocks?
- What does it mean to compose narrative arc computationally?
- What is a pattern? When does a pattern cease to be a pattern?
- Can patterns be emotional?
- Notes
####Homework 1: Create your own human "chamber music" piece.
- Duration: 1-7 minutes
- Collect 5-6 patterns from the "real world" to contribute to your group. They can be images, sound, video clips, objects, etc. Document your patterns in your blog post and write 1-2 sentences on the emotional impact of each pattern.
- The piece does not have to be sound-based, but it must be a live performance.
- You can make use of mechanical devices: metronomes, lightswitches, fans, etc...
- You may want to consider having a conductor, but that is not required.
- Rehearse your piece in front of at least 2 other people.
- Document the rehearsal on video.
- Submit a link to your blog post (you only need 1 per group) to the Homework Wiki
Week 2: Ratios of Space, Color and Time | Code
- Ratios of space.
- Rations of color.
- Ratios of time.
- [Notes] (https://github.com/mimiyin/coding-for-emotional-impact/wiki/Week-2)
- Review of PVectors: Nature of Code Videos 1.1 - 1.4
- PVector Class | Bouncing Ball
- Color wheel
- Use some of the techniques we went over in class 2.
- Show each to a friend.
- Post them to your blog and dropbox and write 1-2 sentences about what worked, what didn't work.
- Failures more interesting than successes!
- Post link to blog and dropbox folder on the Homework Wiki
Weeks 3: The Ebb and Flow of Micro Arcs | Code
- Patterns that swell with anticipation, crest and die off.
- Review Basic Trigonometry: Khan Video | Nature of Code Video 3.2
- Introduction to trigonometric waves
- Modulating scale and speed of patterns
- Applying wave functions to create patterns of movement and time
- Notes
- Install MIDI BUS library
HW 3: Part 1 Due Mon Feb 17th | Post to HW Wiki
- Take 1 of your 3 studies from last week and iterate on it by aiming for the physical characteristics assigned to it by your partner in class. See notes for the complete list of characteristics with links to examples.
- Post to your blog and write 1-2 sentences about what changed.
HW 3: Part 2 Due Sun Feb 23rd | Post to HW Wiki
- Create 3 micro-arcs (< 10s each). Document them. Write 1-2 sentences to describe each micro-arc.
- Choose 1 to loop for 10 minutes. Write 1-2 sentences to describe the pattern at 10s, 30s, 2 minutes, 5 minutes, and 10 minutes. What changes?
- Decoding wave functions
- Non-organic wave functions: Square and Sawtooth
- Notes
HW 4: Due Sun Mar 2nd | Post to HW Wiki
- Create a composition that lasts 1 minute in code using at least 3 different micro-arcs
- Should have a beginning - middle - end
- Write 1-2 sentences describing the narrative arc you're going for
- Making micro-arcs more dynamic
- Modulating wave functions with
- New wave types: Square and Sawtooth
- Recursive drawing
- Creating "patterns" of random()
- Wiggling around with noise
- Hooking up noise with micro-arcs
- Notes
HW 5: Due Sun Mar 10th | Post to HW Wiki
- Make something using recursion or controlling random or noise. Write 1-2 sentences describing what you're going for. Come in comfortable clothing next Monday!
- What's the difference between Timing and Pacing?
- Mapping tempos to "emotional states"
- Approaches to structuring patterns of pacing (tempo)
- Timing the transition between order and chaos
- Manufacturing surprise
- Notes
HW 6: Due Sun Mar 23th | Post to HW Wiki
- Pacing or Tempo: How quickly is the pattern repeating?
- Timing: Ways to think about timing...
- When does the pattern change?
- When does order turn into chaos turn into order?
- How long do you have to do one thing, for the next thing to be surprising?
- How big is the change?
- Decode a video clip or piece of music you love.
- Code one of the patterns we decoded in class. (Notes coming soon.)
- Revisit something you’ve done and tighten the timing and pacing to bring it closer to the emotional arc you’re going for.
- What tempos you used (in beats-per-minute).
- What are the effects of the timing and pacing choices you made (e.g. building suspense, calming effect, fake-out, disorientation)
Here’s a basic timing control example using frameCount
- Playing with the rate of change as a curve
- Linear versus geometric versus exponential growth.
- Notes
HW 7 Due Sun Mar 30th | Post to HW Wiki
- Play with timing curves.
- A polyphonic model for composing narrative and emotional arc. | Notes
- Scoring for Momentous Occasions and Catastrophic Events
HW 8 Due Sun Apr 7 | Post to HW Wiki
- For all options, write 1-2 sentences describing the narrative you're going for. Option 1: Use the keyboard and mouse to create compositions out of the examples from class.
- Goal is honing compositional skills using the knobs and dials of turning voices on and off and controlling their wave patterns.
- Make a storm. | Code
- Make the circle dance. | Code
- Make a poem. | Flash-y version | Timed version
- Record your composition with screen capture.
- Record the keyboard interactions you used to create your composition, like a score so someone else could repeat it.
Option 2: Customize the examples from class.
- Replace the storm code with your own classes: Make a farmyard, make a galaxy, make a dance party of monsters.
- Change up the circle, turn it into a rectangle or hook it up to one of your previous sketches.
- Change the array of strings to create a "poem" with your own content. Use the numbers coming out of the Voice objects to control a different aspect of the words:
- e.g. alpha to fade the words in and out
- e.g. relative ordering of the words in space
- e.g. spell the words out one letter at a time and use the voices to control how many letters to spell out
- e.g. whatever other crazy thing you can think of!
Option 3: Roll your own. Create your own polyphonic compositions from scratch.
- Check out Chorus for an example of how to make the voices change and turn on and off on auto-pilot (requires SoundCipher)
HW 9 Due Mon Apr 21 | Post to HW Wiki
- Create something polyphonic
- Prepare a 5 minute presentation for your final project proposal including:
- Title
- Environment: A description of the physical environment.
- Audience: A description of the person who's experiencing your composition.
- Narrative arc: Provide a verbal description as well as graphs.
- Aaron, Wyna, Kate, Laura, Susane, Federico, Ken, Alejandro
- Prepare a 5-10 minute presentation.
- What was your original idea? Where has it evolved to after a week? What have you discarded? What have you changed? What have you added? How has your “narrative” developed?
- Show something working!
- What have you tried?
- What did you user test? What were the results?
- Show a couple of iterations of the work you’ve done. What works about them? What doesn’t?
- What’s next?
- What are you going to try next?
- What do you think you’ll have by the end of the 2nd week?
- Any questions for the class?
- Technical: How should I approach…
- Conceptual: What’s more effective…
- Please be sure to post documentation of your progress to the HW Wiki so we can zip through your work in a lightning round at the end of class. We’ll have about 5 minutes to spend on each person.
- Neva, Karam, Zoe, Aankit, Michelle, Sagar, Sam, Regina
- Prepare a 5-10 minute presentation.
- Demo what you’ve done!
- Walk us through your narrative. What are your favorite parts. What’s not quite right yet?
- Tell us how you did it! Visual aids, diagrams are always helpful.
- How’d you get here? Did you user test? What did you try? What did you throw away, what did you keep? Show us earlier iterations.
- Those of you who presented last week, please post a link to video documentation of your final so we can do a round of rapid-fire viewing at the end of class.
============================
- 40% Attendance and Class Participation
- 50% Quality of Assignments
- 10% Final Project
Coming out of nowhere. Monotonous drone. Running out of steam. Pregnant pause. Unbearable build up in tension. Gratifying resolution. We use these phrases all the time to describe music and film, animation and dialogue. We strive to provoke these reactions when we compose a piece of music or author a story.
What are we actually talking about? Narrative and emotional arc and its more abstract cousin, composition.
But what does it mean to compose a story or a piece of music in terms of rules and patterns expressed in code? In this class, we will look at how to use simple computational strategies to compose patterns of events capable of producing emotional impact.
The class is agnostic to medium. "Narrative events" can be shifts in color in a painting, shifts in meter in music or shifts in the emotional state of the protagonist of an interactive story. You define the events. The class provides the techniques for shaping those events in dynamic ways over time.
The class is also as much an exercise in cultivating personal sensitivity to the emotional power of patterns as it is a class in how to write code to generate such patterns. We will devote as much time to collecting and dissecting compelling examples of composition as we do to writing code.
Assignments will alternate between programming exercises done in Processing and more creative assignments applying newly learned techniques to the medium of your choice, in the programming language of your choice. Since this is as much a course about perception as it is about code, user testing your creations will be as important as the creations themselves!