Re-apply r267784, r267824 and r267830.

I have updated the compiler-rt tests.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@267903 91177308-0d34-0410-b5e6-96231b3b80d8

Author: pcc

docs/ControlFlowIntegrity.rst

@@ -25,13 +25,25 @@ As currently implemented, all schemes rely on link-time optimization (LTO);
 so it is required to specify ``-flto``, and the linker used must support LTO,
 for example via the `gold plugin`_.
 
-To allow the checks to be implemented efficiently, the program must be
-structured such that certain object files are compiled with CFI
+To allow the checks to be implemented efficiently, the program must
+be structured such that certain object files are compiled with CFI
 enabled, and are statically linked into the program. This may preclude
-the use of shared libraries in some cases. Experimental support for
-:ref:`cross-DSO control flow integrity <cfi-cross-dso>` exists that
-does not have these requirements. This cross-DSO support has unstable
-ABI at this time.
+the use of shared libraries in some cases.
+
+The compiler will only produce CFI checks for a class if it can infer hidden
+LTO visibility for that class. LTO visibility is a property of a class that
+is inferred from flags and attributes. For more details, see the documentation
+for :doc:`LTO visibility <LTOVisibility>`.
+
+The ``-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}`` flags
+require that a ``-fvisibility=`` flag also be specified. This is because the
+default visibility setting is ``-fvisibility=default``, which would disable
+CFI checks for classes without visibility attributes. Most users will want
+to specify ``-fvisibility=hidden``, which enables CFI checks for such classes.
+
+Experimental support for :ref:`cross-DSO control flow integrity
+<cfi-cross-dso>` exists that does not require classes to have hidden LTO
+visibility. This cross-DSO support has unstable ABI at this time.
 
 .. _gold plugin: http://llvm.org/docs/GoldPlugin.html
 
@@ -233,11 +245,6 @@ A :doc:`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
 source files, functions and types using the ``src``, ``fun`` and ``type``
 entity types.
 
-In addition, if a type has a ``uuid`` attribute and the blacklist contains
-the type entry ``attr:uuid``, CFI checks are suppressed for that type. This
-allows all COM types to be easily blacklisted, which is useful as COM types
-are typically defined outside of the linked program.
-
 .. code-block:: bash
 
     # Suppress checking for code in a file.
@@ -247,8 +254,6 @@ are typically defined outside of the linked program.
     fun:*MyFooBar*
     # Ignore all types in the standard library.
     type:std::*
-    # Ignore all types with a uuid attribute.
-    type:attr:uuid
 
 .. _cfi-cross-dso:
 
@@ -260,6 +265,11 @@ flow integrity mode, which allows all CFI schemes listed above to
 apply across DSO boundaries. As in the regular CFI, each DSO must be
 built with ``-flto``.
 
+Normally, CFI checks will only be performed for classes that have hidden LTO
+visibility. With this flag enabled, the compiler will emit cross-DSO CFI
+checks for all classes, except for those which appear in the CFI blacklist
+or which use a ``no_sanitize`` attribute.
+
 Design
 ======
 

docs/LTOVisibility.rst

@@ -0,0 +1,111 @@
+==============
+LTO Visibility
+==============
+
+*LTO visibility* is a property of an entity that specifies whether it can be
+referenced from outside the current LTO unit. A *linkage unit* is a set of
+translation units linked together into an executable or DSO, and a linkage
+unit's *LTO unit* is the subset of the linkage unit that is linked together
+using link-time optimization; in the case where LTO is not being used, the
+linkage unit's LTO unit is empty. Each linkage unit has only a single LTO unit.
+
+The LTO visibility of a class is used by the compiler to determine which
+classes the virtual function call optimization and control flow integrity
+features apply to. These features use whole-program information, so they
+require the entire class hierarchy to be visible in order to work correctly.
+
+If any translation unit in the program uses either of the virtual function
+call optimization or control flow integrity features, it is effectively an
+ODR violation to define a class with hidden LTO visibility in multiple linkage
+units. A class with public LTO visibility may be defined in multiple linkage
+units, but the tradeoff is that the virtual function call optimization and
+control flow integrity features can only be applied to classes with hidden LTO
+visibility. A class's LTO visibility is treated as an ODR-relevant property
+of its definition, so it must be consistent between translation units.
+
+In translation units built with LTO, LTO visibility is based on symbol
+visibility or, on the Windows platform, the dllimport and dllexport
+attributes. When targeting non-Windows platforms, classes with a visibility
+other than hidden visibility receive public LTO visibility. When targeting
+Windows, classes with dllimport or dllexport attributes receive public LTO
+visibility. All other classes receive hidden LTO visibility. Classes with
+internal linkage (e.g. classes declared in unnamed namespaces) also receive
+hidden LTO visibility.
+
+A class defined in a translation unit built without LTO receives public
+LTO visibility regardless of its object file visibility, linkage or other
+attributes.
+
+This mechanism will produce the correct result in most cases, but there are
+two cases where it may wrongly infer hidden LTO visibility.
+
+1. As a corollary of the above rules, if a linkage unit is produced from a
+   combination of LTO object files and non-LTO object files, any hidden
+   visibility class defined in both a translation unit built with LTO and
+   a translation unit built without LTO must be defined with public LTO
+   visibility in order to avoid an ODR violation.
+
+2. Some ABIs provide the ability to define an abstract base class without
+   visibility attributes in multiple linkage units and have virtual calls
+   to derived classes in other linkage units work correctly. One example of
+   this is COM on Windows platforms. If the ABI allows this, any base class
+   used in this way must be defined with public LTO visibility.
+
+Classes that fall into either of these categories can be marked up with the
+``[[clang::lto_visibility_public]]`` attribute. To specifically handle the
+COM case, classes with the ``__declspec(uuid())`` attribute receive public
+LTO visibility. On Windows platforms, clang-cl's ``/MT`` and ``/MTd``
+flags statically link the program against a prebuilt standard library;
+these flags imply public LTO visibility for every class declared in the
+``std`` and ``stdext`` namespaces.
+
+Example
+=======
+
+The following example shows how LTO visibility works in practice in several
+cases involving two linkage units, ``main`` and ``dso.so``.
+
+.. code-block:: none
+
+    +-----------------------------------------------------------+  +----------------------------------------------------+
+    | main (clang++ -fvisibility=hidden):                       |  | dso.so (clang++ -fvisibility=hidden):              |
+    |                                                           |  |                                                    |
+    |  +-----------------------------------------------------+  |  |  struct __attribute__((visibility("default"))) C { |
+    |  | LTO unit (clang++ -fvisibility=hidden -flto):       |  |  |    virtual void f();                               |
+    |  |                                                     |  |  |  }                                                 |
+    |  |  struct A { ... };                                  |  |  |  void C::f() {}                                    |
+    |  |  struct [[clang::lto_visibility_public]] B { ... }; |  |  |  struct D {                                        |
+    |  |  struct __attribute__((visibility("default"))) C {  |  |  |    virtual void g() = 0;                           |
+    |  |    virtual void f();                                |  |  |  };                                                |
+    |  |  };                                                 |  |  |  struct E : D {                                    |
+    |  |  struct [[clang::lto_visibility_public]] D {        |  |  |    virtual void g() { ... }                        |
+    |  |    virtual void g() = 0;                            |  |  |  };                                                |
+    |  |  };                                                 |  |  |  __attribute__(visibility("default"))) D *mkE() {  |
+    |  |                                                     |  |  |    return new E;                                   |
+    |  +-----------------------------------------------------+  |  |  }                                                 |
+    |                                                           |  |                                                    |
+    |  struct B { ... };                                        |  +----------------------------------------------------+
+    |                                                           |
+    +-----------------------------------------------------------+
+
+We will now describe the LTO visibility of each of the classes defined in
+these linkage units.
+
+Class ``A`` is not defined outside of ``main``'s LTO unit, so it can have
+hidden LTO visibility. This is inferred from the object file visibility
+specified on the command line.
+
+Class ``B`` is defined in ``main``, both inside and outside its LTO unit. The
+definition outside the LTO unit has public LTO visibility, so the definition
+inside the LTO unit must also have public LTO visibility in order to avoid
+an ODR violation.
+
+Class ``C`` is defined in both ``main`` and ``dso.so`` and therefore must
+have public LTO visibility. This is correctly inferred from the ``visibility``
+attribute.
+
+Class ``D`` is an abstract base class with a derived class ``E`` defined
+in ``dso.so``.  This is an example of the COM scenario; the definition of
+``D`` in ``main``'s LTO unit must have public LTO visibility in order to be
+compatible with the definition of ``D`` in ``dso.so``, which is observable
+by calling the function ``mkE``.

docs/UsersManual.rst

@@ -1056,17 +1056,8 @@ are listed below.
 .. option:: -fwhole-program-vtables
 
    Enable whole-program vtable optimizations, such as single-implementation
-   devirtualization and virtual constant propagation. Requires ``-flto``.
-
-   By default, the compiler will assume that all type hierarchies are
-   closed except those in the ``std`` namespace, the ``stdext`` namespace
-   and classes with the ``__declspec(uuid())`` attribute.
-
-.. option:: -fwhole-program-vtables-blacklist=path
-
-   Allows the user to specify the path to a list of additional classes to
-   blacklist from whole-program vtable optimizations. This list is in the
-   :ref:`CFI blacklist <cfi-blacklist>` format.
+   devirtualization and virtual constant propagation, for classes with
+   :doc:`hidden LTO visibility <LTOVisibility>`. Requires ``-flto``.
 
 .. option:: -fno-assume-sane-operator-new
 

docs/index.rst

@@ -31,6 +31,7 @@ Using Clang as a Compiler
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
+   LTOVisibility
    SafeStack
    Modules
    MSVCCompatibility

include/clang/Basic/Attr.td

@@ -1611,6 +1611,12 @@ def WeakRef : InheritableAttr {
   let Documentation = [Undocumented];
 }
 
+def LTOVisibilityPublic : InheritableAttr {
+  let Spellings = [CXX11<"clang", "lto_visibility_public">];
+  let Subjects = SubjectList<[Record]>;
+  let Documentation = [LTOVisibilityDocs];
+}
+
 def AnyX86Interrupt : InheritableAttr, TargetSpecificAttr<TargetAnyX86> {
   // NOTE: If you add any additional spellings, ARMInterrupt's,
   // MSP430Interrupt's and MipsInterrupt's spellings must match.

include/clang/Basic/AttrDocs.td

@@ -2380,3 +2380,10 @@ The ``ifunc`` attribute may only be used on a function declaration.  A function
 Not all targets support this attribute.  ELF targets support this attribute when using binutils v2.20.1 or higher and glibc v2.11.1 or higher.  Non-ELF targets currently do not support this attribute.
   }];
 }
+
+def LTOVisibilityDocs : Documentation {
+  let Category = DocCatType;
+  let Content = [{
+See :doc:`LTOVisibility`.
+  }];
+}

include/clang/Driver/CC1Options.td

@@ -282,6 +282,9 @@ def fprofile_instrument_path_EQ : Joined<["-"], "fprofile-instrument-path=">,
 def fprofile_instrument_use_path_EQ :
     Joined<["-"], "fprofile-instrument-use-path=">,
     HelpText<"Specify the profile path in PGO use compilation">;
+def flto_visibility_public_std:
+    Flag<["-"], "flto-visibility-public-std">,
+    HelpText<"Use public LTO visibility for classes in std and stdext namespaces">;
 
 //===----------------------------------------------------------------------===//
 // Dependency Output Options

include/clang/Driver/Options.td

@@ -1152,9 +1152,6 @@ def fwhole_program_vtables : Flag<["-"], "fwhole-program-vtables">, Group<f_Grou
   Flags<[CC1Option]>,
   HelpText<"Enables whole-program vtable optimization. Requires -flto">;
 def fno_whole_program_vtables : Flag<["-"], "fno-whole-program-vtables">, Group<f_Group>;
-def fwhole_program_vtables_blacklist_EQ : Joined<["-"], "fwhole-program-vtables-blacklist=">,
-  Group<f_Group>, Flags<[CC1Option]>,
-  HelpText<"Path to a blacklist file for whole-program vtable optimization">;
 def fwrapv : Flag<["-"], "fwrapv">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Treat signed integer overflow as two's complement">;
 def fwritable_strings : Flag<["-"], "fwritable-strings">, Group<f_Group>, Flags<[CC1Option]>,

include/clang/Frontend/CodeGenOptions.def

@@ -187,6 +187,10 @@ CODEGENOPT(EmitLLVMUseLists, 1, 0) ///< Control whether to serialize use-lists.
 CODEGENOPT(WholeProgramVTables, 1, 0) ///< Whether to apply whole-program
                                       ///  vtable optimization.
 
+/// Whether to use public LTO visibility for entities in std and stdext
+/// namespaces. This is enabled by clang-cl's /MT and /MTd flags.
+CODEGENOPT(LTOVisibilityPublicStd, 1, 0)
+
 /// The user specified number of registers to be used for integral arguments,
 /// or 0 if unspecified.
 VALUE_CODEGENOPT(NumRegisterParameters, 32, 0)

include/clang/Frontend/CodeGenOptions.h

@@ -199,9 +199,6 @@ class CodeGenOptions : public CodeGenOptionsBase {
   /// \brief A list of all -fno-builtin-* function names (e.g., memset).
   std::vector<std::string> NoBuiltinFuncs;
 
-  /// List of blacklist files for the whole-program vtable optimization feature.
-  std::vector<std::string> WholeProgramVTablesBlacklistFiles;
-
 public:
   // Define accessors/mutators for code generation options of enumeration type.
 #define CODEGENOPT(Name, Bits, Default)

lib/CodeGen/CGClass.cpp

@@ -2489,7 +2489,7 @@ void CodeGenFunction::EmitBitSetCodeForVCall(const CXXRecordDecl *RD,
                                              llvm::Value *VTable,
                                              SourceLocation Loc) {
   if (CGM.getCodeGenOpts().WholeProgramVTables &&
-      !CGM.IsBitSetBlacklistedRecord(RD)) {
+      CGM.HasHiddenLTOVisibility(RD)) {
     llvm::Metadata *MD =
         CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
     llvm::Value *BitSetName =
@@ -2565,7 +2565,12 @@ void CodeGenFunction::EmitVTablePtrCheck(const CXXRecordDecl *RD,
                                          llvm::Value *VTable,
                                          CFITypeCheckKind TCK,
                                          SourceLocation Loc) {
-  if (CGM.IsBitSetBlacklistedRecord(RD))
+  if (!CGM.getCodeGenOpts().SanitizeCfiCrossDso &&
+      !CGM.HasHiddenLTOVisibility(RD))
+    return;
+
+  std::string TypeName = RD->getQualifiedNameAsString();
+  if (getContext().getSanitizerBlacklist().isBlacklistedType(TypeName))
     return;
 
   SanitizerScope SanScope(this);

lib/CodeGen/CGVTables.cpp

@@ -902,34 +902,43 @@ void CodeGenModule::EmitDeferredVTables() {
   DeferredVTables.clear();
 }
 
-bool CodeGenModule::NeedVTableBitSets() {
-  return getCodeGenOpts().WholeProgramVTables ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIVCall) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFINVCall) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIDerivedCast) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIUnrelatedCast);
-}
+bool CodeGenModule::HasHiddenLTOVisibility(const CXXRecordDecl *RD) {
+  LinkageInfo LV = RD->getLinkageAndVisibility();
+  if (!isExternallyVisible(LV.getLinkage()))
+    return true;
 
-bool CodeGenModule::IsBitSetBlacklistedRecord(const CXXRecordDecl *RD) {
-  std::string TypeName = RD->getQualifiedNameAsString();
-  auto isInBlacklist = [&](const SanitizerBlacklist &BL) {
-    if (RD->hasAttr<UuidAttr>() && BL.isBlacklistedType("attr:uuid"))
-      return true;
+  if (RD->hasAttr<LTOVisibilityPublicAttr>() || RD->hasAttr<UuidAttr>())
+    return false;
 
-    return BL.isBlacklistedType(TypeName);
-  };
+  if (getTriple().isOSBinFormatCOFF()) {
+    if (RD->hasAttr<DLLExportAttr>() || RD->hasAttr<DLLImportAttr>())
+      return false;
+  } else {
+    if (LV.getVisibility() != HiddenVisibility)
+      return false;
+  }
 
-  return isInBlacklist(WholeProgramVTablesBlacklist) ||
-         ((LangOpts.Sanitize.has(SanitizerKind::CFIVCall) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFINVCall) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFIDerivedCast) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFIUnrelatedCast)) &&
-          isInBlacklist(getContext().getSanitizerBlacklist()));
+  if (getCodeGenOpts().LTOVisibilityPublicStd) {
+    const DeclContext *DC = RD;
+    while (1) {
+      auto *D = cast<Decl>(DC);
+      DC = DC->getParent();
+      if (isa<TranslationUnitDecl>(DC->getRedeclContext())) {
+        if (auto *ND = dyn_cast<NamespaceDecl>(D))
+          if (const IdentifierInfo *II = ND->getIdentifier())
+            if (II->isStr("std") || II->isStr("stdext"))
+              return false;
+        break;
+      }
+    }
+  }
+
+  return true;
 }
 
 void CodeGenModule::EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
                                             const VTableLayout &VTLayout) {
-  if (!NeedVTableBitSets())
+  if (!getCodeGenOpts().PrepareForLTO)
     return;
 
   CharUnits PointerWidth =
@@ -938,12 +947,8 @@ void CodeGenModule::EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
   typedef std::pair<const CXXRecordDecl *, unsigned> BSEntry;
   std::vector<BSEntry> BitsetEntries;
   // Create a bit set entry for each address point.
-  for (auto &&AP : VTLayout.getAddressPoints()) {
-    if (IsBitSetBlacklistedRecord(AP.first.getBase()))
-      continue;
-
+  for (auto &&AP : VTLayout.getAddressPoints())
     BitsetEntries.push_back(std::make_pair(AP.first.getBase(), AP.second));
-  }
 
   // Sort the bit set entries for determinism.
   std::sort(BitsetEntries.begin(), BitsetEntries.end(),

lib/CodeGen/CodeGenModule.cpp

@@ -88,9 +88,7 @@ CodeGenModule::CodeGenModule(ASTContext &C, const HeaderSearchOptions &HSO,
       PreprocessorOpts(PPO), CodeGenOpts(CGO), TheModule(M), Diags(diags),
       Target(C.getTargetInfo()), ABI(createCXXABI(*this)),
       VMContext(M.getContext()), Types(*this), VTables(*this),
-      SanitizerMD(new SanitizerMetadata(*this)),
-      WholeProgramVTablesBlacklist(CGO.WholeProgramVTablesBlacklistFiles,
-                                   C.getSourceManager()) {
+      SanitizerMD(new SanitizerMetadata(*this)) {
 
   // Initialize the type cache.
   llvm::LLVMContext &LLVMContext = M.getContext();