Demon Symbols, known in Japanese as Symbol Encounters (シンボルエンカウント*)?, is a term for overworld enemies that trigger a battle upon interacting with them. They are a modern variant of the random encounters.
Shin Megami Tensei V
Shin Megami Tensei V is the first game in the Megami Tensei series to have symbol encounters directly represent their in-battle counterpart in appearance. There, they are referred to as "Demon Symbols."
Specific species of demons will always spawn in specific spawn points, and targeting those is likely to grant a specific horde of encounters, which is pre-determined upon the encounter spawning on the map. Depending on the location of the demon symbol, the types of arrangements that can be obtained from colliding with them also vary. These encounters are reset should the demons despawn or if the game is reloaded.
The arrangement depends on the levels of the enemies, only being able to be allied by demons of lower levels than them, and their areas, where two demons of different regions cannot be encountered together. In the case of dungeons, such as Demon King's Castle and Temple of Eternity, they're considered extensions of Chiyoda and Taito respectively.
Mitama demons function differently: although they spawn in specific locations of their own (always distinguished according to their species,) they will always respawn if the protagonist fails to initiate combat with them and they run away. However, in the case that the protagonist has initiated battle with the Mitama, their respawn rate is always very low. It can be increased by purchasing and applying the DLC revolving around each respective Mitama. The ability can always be toggled in the settings.
Generally, demons in the overworld will target the protagonist upon spotting him, and will attempt to attack him once they're close. However, the behavior can get much more specific than that between certain races of demons. For example, Byakko will be strikingly fast, and will be nearly able to reach the protagonist while he's in a running stance, while other demons may be much slower. It can get even more specific than that, as some demons, such as Daemon, may be found asleep, and running at max speed will wake them up. Some demons may even follow a pre-set path, which they'll only break if they spot the protagonist.
Demons will not react to the environment aside from the protagonist. Their attacks are able to phase through walls, especially if the protagonist and the enemy are separated by a short fence. Flying enemies may still be attacked from below if they're low enough, even if the slash doesn't actually touch the enemy. Demons cannot see through areas that cannot be walked on as if they're invisible walls, such as pits or lakes.
Normally, demons will instead run away from the protagonist if he's 5 levels higher than them, and they'll only attack if they're cornered and have nowhere to run, although some demons are set to guard certain areas and will give chase no matter the level difference. Other demons, such as Mermaid, will always run away and hide, however.
Interacting with Demons
Generally, simply touching the symbol encounters is enough to enter battle mode, but the protagonist may slash them from a distance, triggering it as well. If the protagonist escapes battle without the enemy demons not having died and/or left by their own will, the enemy encounter will still be there, but that and all encounters will remain idle for around 3 seconds, and colliding with them will not trigger battle for that time: only slashing will be able to do so. After this period is over, the demons will behave as normal.
Mitama can also be found when initiating battle with normal enemies. The specific arrangements and the Mitama that can be found are likely pre-determined: for example, a Saki Mitama will always spawn with two Hecatoncheires, and a Kusi Mitama with Loup-garous. Mitama can be dealt with properly if they're found as part of regular encounters, as even if the Mitama fail to be killed, if the party runs away from the battle, they can re-enter the same fight with the Mitama still in place and try again.
After battle is over, demons and the protagonist may slightly alter their positions after interacting with each-other, and it is possible for the enemy to fail to reappear should no spot be found for it; however, it is very likely the demons respawns after the protagonist ends a battle with another encounter.