I also noticed this 'problem'. However; you are starting the first turn in peace and from the second turn you will get war. Once in war, all walls are broken when you have a unit close to it. That means that this system works like its programmed and its not a bug.
It would be more nicely to fix it (so walls that are build the first turn cannot be broken), but thats more a new idea. It has my support though

The only question you have then. How to handle neutral countries that are walled?