The rest were fighting a war in heaven that Castiel apparently won…and then they just disappeared. It's no secret that the angels were never really that interested in the affairs of men and really wanted to bring about the apocalypse, but now that their story arc has passed, the writers seem to have just forgotten about them.
They are like the big elephant in the room. The leviathans are taking over and no one in the supernatural world really seems to care. The Winchesters seem to be the only people actually doing anything about it and that happens about once every five episodes. What do the angels think about this?
These guys are all smitey this and smitey that, with their ability to pop in and out and it takes A LOT to kill them. It’s not like the leviathans can just eat an angel. That’s going to have to create one heck of an acid reflux.
When the leviathan storyline started, the show’s producers said the angels were not going to be around much because their story was told, but can you just ignore them? Now that Cas is back and a vegetable, there aren’t any other loyal or disloyal angels out there wanting to check in on him. Plot hole, party of one.