Is it based on your main character's level,
your group as a whole's level, or as the highest level in your group?

iether your main person or your strongest (can't say razmas my strongest:p )

They're based on your highest-levelled unit's level.

They are largely based on the average level of your party (inactive and active)

It is not the average level of your active or inactive party members that decides the level of the monsters you will fight. I have many level 30-40 characters that are sitting around (ie Orlandu, Agrias, Meliadoul, Rafa and Malak) and my main party is all level 99. The enemies are all level 98 to 99, with a few 95 or so. I dont think that the average of all my characters would average out to this, when i have about 8 people at level 30-40 and 5 at lvl 99. I think it is based off of your character's levels that you choose to fight the battle, or, the highest in your party. Havent experimented enough to differentiate between those two.

Really the random monsters are just based on what your party looks like...but makes it harder. Mostly your highest lv. achieved is pretty much where all the monsters are gonna be at. So basically dont let your party get too unequaled...it could become one hell of a problem. Cuz its not fun to fight 7 Monsters with 999hp and one deadly attack. You'll have to pray to win...that was the hardest problem i had in the game.