Notes of things to consider / perhaps look into more:

Shannon's information theory laws

Many investigators, especially in the behavioral sciences, seem to believe that scientific advance is predominantly inductive and should be inductive. In terms of the diagram, they believe that progress is made by study of the “raw” data, leading to new heuristic concepts. The heuristic concepts are then to be regarded as “working hypotheses” and tested against more data. Gradually, it is hoped, the heuristic concepts will be corrected and improved until at last they are worthy of a place in the list of fundamentals. About fifty years of work in which thousands of clever men have had their share have, in fact, produced a rich crop of several hundred heuristic concepts, but, alas, scarcely a single principle worthy of a place in the list of fundamentals.

in scientific research you start from two beginnings, each of which has its own kind of authority: the observations cannot be denied, and the fundamentals must be fitted. You must achieve a sort of pincers maneuver.

If you are surveying a piece of land, or mapping the stars, you have two bodies of knowledge, neither of which can be ignored. There are your own empirical measurements on the one hand and there is Euclidean geometry on the other. If these two cannot be made to fit together, then either the data are wrong or you have argued wrongly from them or you have made a major discovery leading to a revision of the whole of geometry.

Induction: the inference of a general law from particular instances: the admission that laws of nature cannot be established by induction. Often contrasted with deduction.

• (induction of) the production of facts to prove a general statement.
• (also mathematical induction) Mathematics a means of proving a theorem by showing that if it is true of any particular case it is true of the next case in a series, and then showing that it is indeed true in one particular case.