The Elder Srolls OBLIVION help!!?

You could buy a house and put your items in a chest. They won't go away as long as you own it. Or you could cast a spell from mysticism to lower your encumberment. Or finish the Dark brotherhood quests to get the horse and store items in it.