First of all: even brushing with nothing – except the dry brush – is better for dental hygiene than doing nothing at all. Before the invention of the tooth brush people were actively caring for their teeth. One interesting example is found in the "toothbrush tree", which has a number of beneficial attributes.
To address the questions:
Abrasiveness is usually no big problem for the gums. Tooth enamel is another story, but:
Baking soda as an abrasive in toothpastes: Mechanism of action and safety and effectiveness considerations:
On the basis of the collected evidence, baking soda has an intrinsic low-abrasive nature because of its comparatively lower hardness in relation to enamel and dentin. Baking soda toothpastes also may contain other ingredients, which can increase their stain removal effectiveness and, consequently, abrasivity.
Even those formulations have abrasivity well within the safety limit regulatory agencies have established and, therefore, can be considered safe.
For plastic retainers after rinsing in your mouth? Totally inert.
If you do not have a regular toothpaste, then baking soda is a nice alternative. But fluoride containing pastes are usually considered way superior. For "remineralisation": tooth enamel is mainly built from hydroxylapatite. Sodium bicarbonate is usually very low in Ca10(PO4)6(OH)2, that is calcium, phosphorous, or beneficial trace elements like strontium and fluoride.
The direct contribution to remineralisation has to be considered zero. The neutralising effect of the soda
(Evidence for biofilm acid neutralization by baking soda) might contribute a little bit to enable the natural capacity of your saliva to remineralise.
Test the baking soda on any piece of plastic, like a bowl or a cup. It will not do anything to it.
That might look very safe and effective now. Reading certain studies just emphasises this:
Stain removal and whitening by baking soda dentifrice: A review of literature:
The evidence available in the literature indicates that baking soda-based dentifrices are effective and safe for tooth stain removal and consequently whitening. A number of clinical studies have also shown that baking soda-based dentifrices are more effective in stain removal and whitening than some non-baking soda-containing dentifrices with a higher abrasivity. So far, research efforts have mainly focused on stain removal and tooth-whitening efficacy and clinical safety of baking soda dentifrices used with manual toothbrushes, with only a few studies investigating their effects using powered toothbrushes, for which further research is encouraged.
As part of a daily oral hygiene practice, baking soda-based dentifrice is a desirable, alternative or additional measure for tooth stain removal and whitening.
But this should not be misread!
All of the above just says that the plan from the question is not really dangerous. Baking soda and xylitol may even be a quite clever combination compared to soda alone.
Comparing toothpaste on this basis with regular paste containing fluoride shows the superiority of added fluoride for maintaining oral health in numerous studies, for example:
Comparing three toothpastes in controlling plaque and gingivitis: A 6-month clinical study:
After 6 months, subjects assigned to the triclosan/copolymer/fluoride group exhibited statistically significant reductions in gingival index scores and plaque index scores as compared to subjects assigned to the herbal/bicarbonate group by 35.4% and 48.9%, respectively. There were no statistically significant differences in gingival index and plaque index between subjects in the herbal/ bicarbonate group and those in the fluoride group. The triclosan/copolymer/fluoride dentifrice was statistically significantly more effective in reducing gingivitis and dental plaque than the herbal/bicarbonate dentifrice, and this difference in efficacy was clinically meaningful.
That means unless your local drinking water is very high in fluoride or you drink large amounts of green or black tea (containing large amounts of fluoride) tooth pastes with fluoride will be much better at protecting dental health.