This is not the type of question I have ever considered . . . so I asked my wife. She happens to be a college physics professor so . . .
We looked at bouancy, density and all sorts of things.
The short answer is . . . doable but risky. It would sink fairly low in the water which would help, but is likely to still be tippy. If it tipped it would completely roll. So four people inside (Guessing here since that seems the best way to balance people) would need to stay really well balanced. And your weights are not the same. How are you planning to power it across the river? Doing something like paddling would put someone up near the edge and reaching over - increasing the liklihood of tippage. And there is a current to deal with.
I give a lot of points for creativity so I would definitely give it a chance to work. But you would be making saves of some sort. And if they fail you are wet. With whatever consequences that brings. Also, it is extremely unlikley that you would get across dry. Water splashing in wouldn't sink you but it would be cold and wet.
I can envision some things that might make it safer, but none eliminate that reasonably high chance of tipping. .