A Max/MSP newbie muddling through the of subjective testing for audio! This is the third of three posts which cover the test design for my research project. Click for one and two.

Hello all,

Dare I say it, everything looks to be very close to finished in terms of the testing interface and crucially, the method of collecting and sifting through the results! This post is going to take you through the testing procedure and processing of results. Hopefully everything will make sense, if not then thankfully I can catch it before the actual testing starts!

Collecting Results

This test section allows participants to play around and learn the interface they will be using for the test proper.

The first thing participants will see after the section to input details such as age, will be the test section. Here users can familiarise themselves with the interface. A totally separate surround sound recording will be playable from this section. Once they are comfortable, they can move on to the test proper.

The picture above is what participants will see at the end of the test. There are 12 sections so there is no need to post all that. The important thing is the test reference number, which is the main point behind this blog post. Previously, I talked about my “Randomisatron”. This messy, complex and head melting monster of a subpatch allows me to shuffle the standard order of the test. The randomisation allows my test to meet the ITU BS1116 standard for the playback of test material where test samples must be presented in random order for each test participant.

Since I know the default playback order for each section (A and B, C and D, E and F etc.) I can shuffle the playback order with a code using the Randomisatron (this name may stick, but maybe not for the write up) With the participants answers filled out, I should be able to (un)shuffle the answers back into the original order with reference to that randomising code. This calls for a trip to Excel!

Processing the Results

My friends have always slagged me off because of how much I like using Excel. That is not to say that I spend my evenings pondering some Excel magic. The reason I like Excel is because at the end of this test, I am going to have 5 answers for each of the 12 sections from at least 20 participants. This is not something I want to do on paper. Excel appeals to my lazy side and if I can get rid of a lot of leg work with one swift click of the mouse, I will gladly take it!

Test Ref — The test reference code contains the randomising information.

Bear in mind that the Excel file is made so I understand it where the test interface has to be much more sleek. Hopefully I can make this make sense for you! The two columns on the left are the default order which the whole test interface has been built around. This info also pops up on the far right as a reference to make things a bit easier. What I have highlighted in yellow is the randomising part of it.

When I get a set of results from a test participant, I will fill in all the answers for questions 1 to 5 all the way down the 12 sections. These answers can only be A or B/1 or 2, the numbers I have in at the moment were just for when I was testing the thing out. With the answers filled in, I will then take a look at the test reference number and type it into the left hand yellow column. The right hand yellow column is a copy of what the default is, 1 through to 12.

Here is that first Excel image again. Imagine there are 12 CDs with the recordings on them. Take a look at the first CD which is “SF1, T1”. The randomising code is telling the Max patch to play CD1 2nd in the queue. It is telling the second CD to play 12th in the queue and the third CD to play 1st.

The first step is to sort the New Question Order column by the Rand. column. This turns the Rand. into 1 to 12 while also moving the New Question Order numbers with it. What this does is match up the numbers in the New Question Order with the answers I filled in at the start. Remember earlier I said that the third CD was being told to play 1st? Take a look at the first number in the orange column. Remember that the first CD is to be played 2nd in the queue? You can see that from the second spot in the orange column too. Finally, you can also see that the 2nd CD was played in the 12th position. Notice how the CD1 and CD2 answers are A, B, C and D in the image. We know that these below to the first two sets, recordings 1 to 4, so they ideally should be at the very top and lined up with their “SF1, T1” counterparts at the far left of the spread sheet.

BOOM!

I do these steps for each participant and when I get all 20+ finished I can then sort all the Question/Answer columns by the Original Order column and now all the answers for each set of recordings are grouped together, ready for me to do some whole other amount of work so I can get statistics and results processed, something I best read up on soon.

Thanks!

Thanks again for reading. It looks like I have made a small trilogy of posts about how to administer a subjective test in terms of playing sounds to meet the ITU standard, collect the answers to the questions and then process them for ease of use later on. Take a look at the top of this post for links to the other posts.

Thanks very much for reading, I hope someone finds it helpful!

Hi all,

I was chatting with a friend recently and the meme below came up. A few minutes ago, it hit me that the meme could be a good way to structure a post or set of posts about my MSc project, that and it means that I don’t have to do this through the medium of interpretive dance!

It has been pointed out to me that my blogs are usually quite long. I agree, I do love the look of my text! Anyway, anyone who could have a passing interest may get pushed away by my blabbing so like a meme, this post is going to describe the project I am doing in a short and sweet way. It is important that I use the blog to keep track of the project in all its technicalities, but I also want to keep an informality to posts at the same time. What better way to do that then through a meme!

Observation

The Soundfield system allows an engineer to replace the 5 microphones normally used for surround sound recording of classical music with a single microphone. The Soundfield microphone allows an engineer to adjust what has been recorded after the recording takes place to create the best sounding production that is possible from the microphone being placed where it is. It is quite easy to setup and use. This is a huge selling point. Imagine taking a photo and wishing you could change the lighting days, months or years after the photo is taken!

A “traditional” surround recording system uses 5 separate microphones, also known as a multi-microphone array. Each microphone has its own stand with each having a specific angle and distance relationship with each other. The rear microphones are generally spaced a fair bit away, sometimes meters away. They can be difficult to set up and if a mistake is made, it can not be fixed after recording, a bit of a hassle and you can’t adjust things after the recording like you can with the Soundfield.

Questions

Well, there are significant physical differences between the Soundfield and traditional arrays. Does the Soundfield sound better or worse to a set of listeners in a specific recording session? Can they both achieve excellent results? If one does and the other doesn’t, why is that? Would the Soundfield rate with listeners given the significant differences between it and traditional arrays? Is the ease of use and after recording touch ups worth any quality issues that the Soundfield may have?

Hypothesis

Well, this is a tough one. A lot of those questions can be answered by saying each recording method has its own characteristics. Certainly, there wouldn’t be one better or one worse across music recording as a whole. But, if you record a musical ensemble and play the results to some listeners and ask them what they prefer, maybe one method would stand out. At this early stage of the project, I would wonder what people think. I think that the two methods are both as good as each other and are viable methods of recording music, but when recording a musical ensemble I would question how listeners would react to how differently the Soundfield deals with the rear microphones.

Prediction

Well, I am not too sure about this one yet. Since my last post, I have been doing a lot of reading and my literature review so far do not show much research directly comparing the traditional arrays vs. the Soundfield for the type of music I want to use (small classical group). So I am on the fence. As an engineer, it would be great to know that the easier to use Soundfield is as preferred if not more preferred than traditional arrays. So, lets find out.

Experiment

Through research of the various Audio Engineering Society papers, I have found that the Decca Tree and its derivative called the Fukada tree surround arrays are two of the most preferred traditional arrays. So, to ensure that the Soundifeld is getting a fair fight as such, I want to record a small classical ensemble with the arrays. To make sure everything is equal, the arrays will be setup simultaneously and recorded at the same time too. Then I want to play extracts to listeners to find out which is the most preferred.

I ruddy hope so!

My next project post will be about what I have found during my literature reviews!

Thanks for reading!

Tag: tree

Surround Sound Subjective Test Design 3 – Collecting the Data

Collecting Results

Processing the Results

Thanks!

My Project Description Take 2 – With a Meme!