What is science, science of security?

Minimally, to count as scientific, we expect a theory to have the following properties:• Consistency: claims are consistent with other claims and available observations. Inconvenient observations are not discarded.• Falsifiability (see above): we can describe the evidence that would prove claims wrong. Without this we are not self-correcting [14].• Predictive power and progress: models and theories should facilitate accurate predictions, and the set of observations that can be accurately predicted should generally
increase over time. We should not be asking all of the
same questions year after year 

可证伪的才是科学。数学不是科学，但科学具有数学性。
而且，越有可能证伪的学说，越有贡献。

scientific method. The idea is to attempt to generalize, making falsifiable statements that are consistent with what we have already observed, but predict also things not yet observed. Then seek new observations, especially those expected to present severe tests of predictions (rather than those expected to corroborate them). Often called the hypothetico-deductive model, the summary is:
1) Form hypotheses from what is observed.
2) Formulate falsifiable predictions from those hypotheses.
3) If new observations agree with the predictions, an hypothesis is supported (but not proved); if they disagree, it is rejected.  

Note that this process of iteratively eliminating possibilities that conflict with observations is the essence of differential diagnosis in medicine, sensible approaches to car repair, and the investigative method Sherlock Holmes recommends to Watson: “Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.”  

III. SCIENCE OF SECURITY 

While many open questions remain in the Philosophy of Science, much is also settled. Far less is settled in discussing Science of Security; what emerges from review of the security literature below is, in many places, an absence of consensus; we return later to consider if this signals an immature science. 

A. Science of Security: Early Search and Misunderstandings

Multics seniors still remind the young that today’s problems are not only 40 years old, but were better addressed by Multics  

He notes, on the challenge of using system models, that properties proven “may or may not hold for the real system depending on the accuracy of the model”, and that for high-level abstractions,What is important is that the abstractions are done in such a way so that when we prove some property about the abstraction, then that property is true of the real, running system。”

B. Science of Security: Recent EffortsHere we selectively review research since 2008 under the label “Science of Security”; our goal is not an encyclopedic review per se, but to provide context for later observations。

The desire to do security more scientifically fits into a larger picture of frustration voiced by experts—e.g., in accepting the Turing award in 2002, Shamir made a set of 10-year predictions that included “the non-crypto part of security will remain a mess.”  

JASON was requested by the DoD to examine the theory and practice of cyber-security, and evaluate
whether there are underlying fundamental principles that would make it possible to adopt a more scientific approach, identify what is needed in creating a science of cyber-security, and recommend specific ways in which scientific methods can be applied..  

The science seems under-developed in reporting experimental results, and consequently in the ability
to use them. The research community does not seem to have developed a generally accepted way of
reporting empirical studies so that people could reproduce the work.  

Security可以是science吗？他和传统science区别在哪？在于他是人工品，不具备客观真相，而且是有对手的，对手会变，所以传统基于现象去提出假说并不适用。

什么是科学？科学的方法是什么？安全是科学？如何将安全研究科学方法化，从而更有效的研究来产生知识，避免产生的知识过于依赖当时的现状，不能传承。（比如科学研究有更有的验证方法。)

crypto(provable security)有的人觉得是科学，有的人说不是，因为crypto并不能保证real system的安全性，还有其他系统实现的东西要考虑。

IV. FAILURES TO APPLY LESSONS FROM SCIENCE

We now detail security research failures to adopt accepted lessons from the history and philosophy of science.

A. Failure to observe inductive-deductive split

Rather, the question is to what degree properties proven about a mathematical system can be  translated into useful properties of a real-world one.  If security is proved in the mathematical sense, then it can’t refer to a realworld property. (e.g. 128bit key more secure than 64bit)

B. Reliance on unfalsifiable claims

For example, to falsify “in order to be secure you must do X” we would have to observe something
secure that doesn’t do X. If we interpret “secure” as a realworld property, such as the avoidance of future harm, then observing it requires knowing the future. On the other hand, if “secure” is interpreted formally, while we can now identify mathematically secure systems, we can make no deductions about real-world events (e.g., that harm will be avoided). A similar argument shows that claims of the form “X improves security” are unfalsifiable.

In summary, claims of necessary conditions for real-world security are unfalsifiable. 

C. Failure to bring theory into contact with observation

A scientific model is judged on the accuracy of its predictions (Section II-C1); lack of data or difficulty in making measurements does not justify trusting a model on the sole basis of its assumptions appearing reasonable. But this is often done in security research.

Community actions were based on the assumed truth of something that depended critically on an untested assumption.  

D. Failure to make claims and assumptions explicit the evidence falsifying a precise claim is easily described. If a theory says “X should never happen under assumptions A, B and C” then showing that it does suffices to refute the claim. But when a statement is vague, or assumptions implicit, it is unclear what, if anything, is ruled out. Thus, difficulty articulating what evidence would falsify a claim suggests implicit assumptions or an imprecise theory [3].  

The problem of implicit assumptions seems widespread. (e.g. less user click on link after notification implies secure, that raising awareness of cyber threats or paying more attention to warnings is

inherently beneficial. )

E. Failure to seek refutation rather than confirmation The limitations of formal approaches noted in Section IV-A might lead to belief that empiricism wins—that measurement and experimentation are the clear way forward for pursuing security scientifically. The truth appears more complex. Recall that in the hypothetico-deductive model (Section II-E), hypotheses are most useful when they allow anticipation of as-yet unseen things, and observations are most useful when they present severe tests to existing hypotheses (vs. simply corroborating existing beliefs). If that model is not to be a random walk, observations must actively seek to refute existing belief (see Section II-D).  

即使理论上验证了，因为实现的不同，不能证明secure。

如果是攻击文章，相当于是新的observation，那最好是能攻破以前以为是secure的方案，而且能通过observation提升出新的generalized的theory。

 

V. WAYS FORWARD: INSIGHTS AND DISCUSSION T1: Pushes for “more science” in security, that rule nothing in or out, are too ambiguous to be effective. Many insights and methods from philosophy of science remain largely unexplored
in security research.  

Recalling Popper’s view that to count as scientific a statement has to “stick its neck out” and be exposed to risk, we suggest that the same is true of pursuing security scientifically: to be effective, calls for more science should specify desired attributes, specific sources of dis-satisfaction with current research, and preferred types of research. 

希望未来的research能不提不可证伪的假说（不要怕被证伪），新的research侧重于提出observation（attack）反驳现有的假说，并且提出新的假说（建立在已有条件下，如果做了其他的什么，就可以保证secure）。

T2: Ignoring the sharp distinction between inductive and deductive statements is a consistent source of confusion in security 。

deductive是抽象推演，inductive是现实的经验总结，两者混到一起了。

T3: Unfalsifiable claims are common in security—and they, along with circular arguments, are used to justify many defensive measures in place of evidence of efficacy  

T4: Claims that unique aspects of security exempt it from practices ubiquitous elsewhere in science are unhelpful and divert attention from identifying scientific approaches that advance security research.  

T5: Physics-envy is counterproductive; seeking “laws of cybersecurity” similar to physics is likely to be a fruitless search  

T6: Crypto-envy is counterproductive; many areas of security, including those involving empirical research, are less amenable to formal treatment or mathematical role models.  

The main point is that despite many pointing to crypto as role-model for a Science of Security, its methods are less suitable for numerous areas, e.g., systems security and others involving empirical research.  

T7: Both theory and measurement are needed to make progress across the diverse set of problems in security research.  

T8: More security research of benefit to society may result if researchers give precise context on how their work fits into full solutions—to avoid naive claims of providing key components, while major gaps mean full-stack solutions never emerge。

T9: Conflating unsupported assertions, and argument-byauthority, with evidence-supported  tatements, is an avoidable error especially costly in security  

“Before the underlying science is developed, engineers often invent rules of thumb and best practices that have proven useful, but may not always work.”  

In summary, scientific statements stand or fall on how they agree with evidence. Calling something a principle, best-practice, rule-of-thumb, or truism removes no burden of providing supporting evidence 。

T10: Despite consensus that assumptions need be carefully detailed, undocumented and implicit  assumptions are common in security research。

Connections between abstractions and the real world (Section II-C) are often unchecked or loose in security。Platt [26] recommends answering either “what experiment would disprove your  hypothesis” or “what hypothesis does your experiment disprove.” 

T11: Science prioritizes efforts at refutation. Empirical work that aims only to verify existing beliefs, but does not suggest new theory or disambiguate possibilities falls short of what science can deliver  

In science, there is an expectation to seek refuting observations, as discussed in Sections II-D, IV-E. Corroborating evidence is never definitive, whereas refuting evidence is.  

VI. CONCLUDING REMARKS 

From the preceding points, some overall observations emerge related directly to Science. A first meta-observation is that the Security community is not learning from history lessons well-known in other sciences. security research is learning neither from other disciplines nor its own literature, and questioning security foundations is not new .

A second meta-observation pertains to those seeing the endgoal of security research being to ultimately improve outcomes in the real world. The failure to validate the mapping of models
and assumptions onto environments and systems in the real world has resulted in losing the connections needed to meet this end-goal. A rigorous proof of security of a mathematical
system allows guarantees about a real-world system only if the coupling between them is equally rigorous. We have seen repeated failure in poor connections between mathematical systems and real-world ones, and consequent failure of the latter to enjoy properties promised by the former. 

That the Security community is experiencing problems historically well-known in other scientific fields is unsurprising—and perhaps even supports claims of being a Science. What is harder to accept is apparent unawareness or inability to better leverage such lessons.  We have noted the absence of consensus in many areas of Security, which some might take as signaling an immature field. 

On a positive note, one point of consensus is that security research is still in early days. Those who pursue a Science of Security should be cognizant of history—including that progress in science is neither steady nor straight-line. Simply wishing for a Science of Security will not make it happen.
What is needed is for security researchers to learn and adopt more scientific methodologies. Specific guidance on what those are, and training in recognizing and using them, may help security research become more scientific  

总结：

科学是为了产生高质量的知识，如果是一门科学就可以使用科学方法。科学方法有很多种，（有逻辑学，集合论，观察法等等）。这篇文章先讨论了怎样的学科能算得上科学，基本上必须要可证伪，这样才可以快速得到知识。科学方法的目的就是快速从已有的现象中提炼出上层的思想理论。并且这些理论要有普适性。

之后这文章说安全现在有可能可以成为一门科学。但是，现在的文章经常犯一些科学不允许的错误，或是使用不科学的方法，比如提出不可证伪的假说，或是用observation（attack）来发文章。科学的方法应该先通过观察提出可证伪的假说，然后通过观察来推翻假说，进而提出自己的假说，提出的新假说应该有期待的反例，越有可能产生反例的假说，越有价值。所以希望以后security的研究，应该运用成熟的科学方法来进行指导，从而快速产生可重复的知识，而不是很快就会被遗忘取代的观察。