Surround Sound Subjective Test Design 2 – Randomisation

A Max/MSP newbie muddling through another aspect of subjective testing for audio, randomisation of playback! This is the second of three posts which cover the test design for my research project. Click for one and three.

Hello!

Exciting topic eh? Ohhh yes! Well, my supervisor was quite happy with the Max/MSP way of doing things so I have been tinkering with the patch, well hacking it to bits and sticking it back together actually.

Quick Note

My friend Michael McLoughlin pointed out that my way of selecting channels by using 6 ezdac~ objects and setting each one to the channel I wanted works but there was a better way! Instead, use the dac~ object and specify the required outputs. 5.1 needs 6 so the following image is what you are left with.

Here we have the recording selection at the top and the much neater "dac~ 1 2 3 4 5 6" to provide 6 output channels.
Here we have the recording selection at the top and the much neater “dac~ 1 2 3 4 5 6” to provide 6 output channels.

Lingo

From reading the draft, I am confusing myself! So hopefully this will help.

Section: The place where two recordings are compared against each other, the questions are given here.

Set: Each section contains a set of 2 recordings which are compared with eachother.

Song: The recording session provided me with 2 songs.

Piece: It was recommended to use two pieces for each song to help establish meaningful results.

Randomisation

With the fundamental surround sound player more or less finalised, I spent some time calculating how many sections were required. In total, there will be 12 sections each of which playing a set of 2 recoding extracts. With this many sections and sounds, there can be an issue with bias towards the earlier material. If I get 20 participants and each of them has the exact same playback, or lets say if the same album is played to each person from start to finish then results may show that people who get bored or tired do not give accurate data towards the end of the test. If things are randomised, so the album is being played in shuffle then each question should get a fair shot at being answered well before any boredom sets in, but no one is going to get bored of course. . . Seriously though, if the test was a long one and the participants ears tended to get tired towards the end then you can basically discount any results from that part of the test as they are unreliable. To address this,  as well as randomising play back I am keeping the test length close to the 20 to 30 min park as per ITU BS1116.

How random is random enough?

The way I am randomising playback is by taking each set of recordings and shuffling them about. In other words, if I have 12 sections which contains a set of 2 recordings in each, then shuffling those 12 sections around should be enough.  If I only shuffled the set of recordings in each section, it wouldn’t stop latter sections being hit by the tiredness/boredom aspect I mentioned as {A, B} would become {B, A} but would always be first to be played no matter what.

The Randomisatron

The lowest ideal number or participants is 20.There are 24 pieces of audio in the test at the moment. Since there are always 2 being assessed at one time, that means there are 12 sets of recordings which means 12 sections of questions. The 12 sections are just a way of playing sounds, with no information inside them regarding what they are playing. Each section is waiting to be told what files to open and playback. If I could devise a way of sending each pair of recordings to one of the 12 sound players in a random way then I should be on a good track.

The Randomisatron
The Randomisatron

This is what I came up with. The numbers on top correspond to the 12 destinations that the recordings could go to, which is one of the 12 sections. By clicking one number from each group, you are routing each set of recordings to different sections. With 12 destination buttons for each set of recordings, you can route them anywhere you like. Clicking 12 through to 1 would reverse the playback order for example. However, there is no way to reverse the set of recordings themselves {A. B} -> {B, A}. This is because it might be over board and more importantly, very complex to keep track of when it comes to calculating the results.

By using the random feature of Excel, I quickly came up with 20 lists of numbers from 1 to 12 inclusive which were randomised. All I have to do in the test situation is input those numbers, take note of the numbers and then compile results into a big spreadsheet (The Unrandomisatron maybe?). This can sort the data for me. The way in which I am exporting the data is by a handy screen grab feature in Max/MSP. There is a lot of potential for human error when I am taking not of the randomising number so I could include a number display that which will tell me exactly how each test was routed. That should clear up a lot of fuss!

p 14

Out of all the boxes Max/MSP boxes you have seen in my blog so far, this one probably makes the least sense. It is a subpatch object. This means that you can tidy up your patch by placing certain elements in a subpacth. It makes things quite handy for duplication of finished patch, this one was copied 12 times! P 14 is just a name I had for some reason, all you need is “p” and a word after counts as its name, if you want one.

The mechanics of it all
The mechanics of it all

The brown numbered boxes at the top are inlets, they are inputs coming from the main patch file which contains the subpatch. Inlets 1 to 12 and 13 to 24 are fed by the 1 to 12 destination buttons from earlier in the post.  Inlets 25 and 26 are fed by the file names for each surround file. These filenames are what are being sent to each section, two files per section. They are commands to tell the sfplay~ objects in each section  to open a specific file.

The gate object routes what is coming in the top right input to any of the outputs at the bottom depending on what number it receives on the left input which in this case is between 1 and 12. Those are then connected to the outlets of the patch in blue at the bottom. Note that I had to add 1 to the numbers coming in to fix a glitch with where the gate was confusing the sfplay~. What this means is that the gate is routing to its own outputs 2 to 13 instead. Anyway, that quirk aside depending on what is number is coming in through the inlets 1 – 24, the signal from inlets 25 and 26 will be sent to the corresponding outlet as can sort of be seen below.

Now that I look at it, I could have just send the signals from inlets 1 to 12 to both of the gate objects inside the subpatch, this would result in the same outcome. Instead, I have the 1 to 12 destination buttons going to both 1 to 12 and 13 to 24 as can be seen below. See how the blue ones go to one half and again to the other in the two images below? Each set of 12 control the destinations of the two final patch cords on the far right of the object which themselves are the open soundfile information for the sfplay~ objects

Random 3Random 4p send is just a subpatch called send. The inputs are simply routed to a set of Send boxes which sends signals to their respective receive boxes which are placed in the middle of the patch for each section. This keeps things nice and neat!

Quick Note on Arrays

To fill you in on what is actually being tested the following arrays have been derived from the recording session

1) Traditional array

2) Soundfield in the ITU surround preset (this means it matches the ITU angles)

3) Soundfield with adjusted angles for the rear pickup in an attempt to get the best sound.

The inclusion of the third “array” allows the big selling point of the Soundfield to be tested. That is that you can adjust things after the recording. So I made some adjustments in the studio the other day, months after the recording which is case in point of that selling point. These adjustments were to make what I think is a better sounding production. This means that the Soundfield may not do very well in its default preset, but once adjusted could do better!

Thanks

As always thanks for reading, this blog is a great resource for me to log my progress and keep my mind on track. Hopefully, any readers will be interested and learn something new! Though, as I keep wanting to point out I am not the most experienced with Max/MSP so be kind!

BTW, log, broadcast. . . broadcast log. . BLOG!?!? Is that where it comes from? Mind blown!

3 thoughts on “Surround Sound Subjective Test Design 2 – Randomisation

Leave a Reply

Your email address will not be published. Required fields are marked *