Yes, non-UK residents can incorporate limited companies in the UK. The registration process for non-resident is exactly the same as if you lived in the UK, so you can set up a company online in just a few hours. One difficulty you may face is getting a business bank account in the UK, but you can open an account in France if you get your company documents legalised for use outside the UK.
You can run your company from France but you will need a UK registered office address. This is where your official correspondence will be delivered. Details of this address will also be on public record. You can get a professional registered office address from a company formation agent or other third-party address provider.
Regardless of where you live, you will have to adhere to UK company law and taxation requirements. This means you will have to register your business with HMRC and pay business taxes in the UK. However, you should be able to claim double taxation relief. You may also have to pay personal Income Tax to HMRC on any money you receive from the company.
Please speak to an experienced accountant before setting up a company in the UK. You need to be fully aware of your obligations before committing to this decision and you will probably require professional help to complete your accounts and tax returns.