Edom was a nation older than Israel. In fact, the Hebrews requested permission to travel through the country of Edom during the Exodus, but they were denied access and had to travel around the kingdom. However, Edom had ties to the Hebrews. Tradition says that while Israel was a nation of people descended from Jacob, Edom was a nation descended from Esau -- Jacob's slightly older twin brother.
This perceived relationship led many Jews to believe that Edom had betrayed them during the capture of Jerusalem, choosing to side with foreign invaders against them. In response, Edom often received special attention from prophets during the exile who warned that God had extra punishments in store for the ancient nation of Edom. In Obadiah, the warning is that Edom should not gloat in Judah's defeat because Judah will one day be restored and will defeat and rule Edom.