If I get a new device and want to put my CPFS Mobile SIM in it, will it start working automatically or do I need to change any settings in the app?

As long as your device is fully unlocked and compatible, once you insert your CPFS Mobile SIM and turn the phone off and on, it should start working as normal!