Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hfo2 update: take 1 #72

Merged
merged 37 commits into from
Feb 3, 2020
Merged

Hfo2 update: take 1 #72

merged 37 commits into from
Feb 3, 2020

Commits on Jan 22, 2020

  1. Hermetic builds inside a container

    Adds 'build/docker/Dockerfile' which describes the base container image of
    Hafnium compilation environment. This image is built and uploaded to GCP
    where users download it from. The feature is always enabled for Kokoro
    and can optionally be enabled for local builds too. Once rootless
    containers are easier to set up, we might make it the default for local
    builds too.
    
    An arbitrary command can be executed inside the container with
    'build/run_in_container.sh [-i] <command> ...'. This is done
    automatically inside 'Makefile' and 'kokoro/ubuntu/build.sh' which
    detect whether they are already running inside the container and respawn
    themselves using 'run_in_container.sh' if not.
    
    The feature is guarded with HAFNIUM_HERMETIC_BUILD environment variable,
    switched on if the value is "true". All other values switch it off, e.g.
    'run_in_container.sh' sets it to 'inside' to avoid recursion.
    
    Bug: 132428451
    Test: HAFNIUM_HERMETIC_BUILD=<value> make
    Test: HAFNIUM_HERMETIC_BUILD=<value> kokoro/ubuntu/build.sh
    Change-Id: I0737a868ab4f67c0fdbf78fa8a97cc91714d2e10
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    c447e03 View commit details
    Browse the repository at this point in the history
  2. Add depfile for linux targets

    Linux tree did not use to specify any sources and would therefore never
    be rebuilt after first build. Add a script which generates a depfile for
    the tree and use it for a target that linux targets depend on.
    
    Test: make ; touch third_party/linux/README; make (should recompile); \
          make (should not recompile)
    Change-Id: Id0a06be73de2ed31c52f1697ac00f7d45425b43b
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    d24b808 View commit details
    Browse the repository at this point in the history
  3. Keep logs in a circular buffer so they can be extracted from a RAM dump.

    Bug: 115484857, 132429213
    Change-Id: I7ae3fd6c24ca8b42f581d9d4417fc6c49b23b43d
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    2535641 View commit details
    Browse the repository at this point in the history
  4. Updating linux submodule

    Change-Id: I7ce6e1c5f540c9a619c70949fb99606c89fbabe9
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    88d9a21 View commit details
    Browse the repository at this point in the history
  5. smc wrapper function conformance

    smc wrapper function accepts all arguments and returns all four result
    registers.
    
    Bug: 132421503
    Change-Id: If3030df8ffe77f26a89d73959d9e05c3bd269c05
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    a8db03e View commit details
    Browse the repository at this point in the history
  6. Fix stack push/pop alignment for SMC wrapper

    AArch64 requires stack pointer alignment of two X (8 byte) registers.
    
    Bug: 132421503
    Change-Id: Ib93bd47a49286ec9646bfd1f0456c8765fa64aca
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    043fa18 View commit details
    Browse the repository at this point in the history
  7. Create separate path for SMC forwarding

    This is the first stage of creating an SMC forwarding path based on
    the manifest.
    
    Currently none of the SMCs are forwarded, which maintains existing behavior.
    
    Bug: 132421503
    Change-Id: I1c3257b83346d47ff1e71db3898b9b85be14a043
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    a8c9385 View commit details
    Browse the repository at this point in the history
  8. Better error handling in Hafnium tests

    When the emulator crashes/times out, instead of generating a json output, hftest
    crashes when a json parsing exception is thrown.  This patch handles the
    exception and tries to print the error it encountered instead.
    
    Change-Id: Ice353e6a53c64ecc2cfdc8045cbde0a1fed2077d
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    7819db4 View commit details
    Browse the repository at this point in the history
  9. Fix capitalisation of VM in comments.

    Change-Id: Ib337ebdebee9cb917029c464f292f1b4cbf11cba
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    a11d189 View commit details
    Browse the repository at this point in the history
  10. Autoformat docs.

    Change-Id: Ibc270de8cd6d8535e20847098175d8a00c67abe9
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    b57a111 View commit details
    Browse the repository at this point in the history
  11. Adding overview of tests, based on David's doc.

    Change-Id: I3e2e262966a0584a43a86a49570d0d3db7cff09f
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    279ab27 View commit details
    Browse the repository at this point in the history
  12. Simplify GN logic by creating a target for prebuilt Linux

    The `initrd` GN template currently supports two different ways of
    specifying a primary VM image - name of a target or a path to a prebuilt
    image. These use different target properties (as GN has no mechanism of
    classifying a string as either a target or a path).
    
    Simplify this in anticipation of using different Linux targets for
    different HW boards in platform args by creating a target for each
    prebuilt image. The `linux_kernel` template accepts a `prebuilt` path
    and creates a "${target_name}__prebuilt" target which copies the image
    into the out folder. As a side effect, this more closely ties the image
    to the source code in the GN file.
    
    Change-Id: I2afa295e666581393e9a80c311d7d8fe56d4fbba
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    34f8a4b View commit details
    Browse the repository at this point in the history
  13. Generate VM IDs with an offset of 1

    The SMCCC reserves VM ID 0 for the hypervisor itself, therefore, other VM IDs
    should start after that (at least, but not necessarily at 1).
    
    - Generate VM IDs with an offset defined by HF_VM_ID_OFFSET (currently 1)
    - Fix hardcoded IDs and other code that assumes VM IDs start at 0.
    
    Bug: 132421503
    Change-Id: I47cff8734ac153dcb1a1a435285153d6caf2877e
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    bef2e52 View commit details
    Browse the repository at this point in the history
  14. Add doc about running tests on FVP.

    Change-Id: I29063eb7a0ef66f9093740c158cff15512ed15d2
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    dfbab38 View commit details
    Browse the repository at this point in the history
  15. Adding documentation on VM interface, and links from README.

    Change-Id: I73e9ce0ff5e4de92f4a86ce78bb49fd6a7f7bda5
    qwandor authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    3c3bc85 View commit details
    Browse the repository at this point in the history
  16. Add script to dump system files used by build

    In order to move towards builds which only use resources in the Hafnium
    repo, this patch adds a script which runs the build with strace and
    dumps all files touched in the process. Files in the Hafnium directory
    and in /tmp are automatically filtered out.
    
    Bug: 132428451
    Test: ./build/strace_open.sh opened_files.txt
    Change-Id: I03a2df4eedf40c456b65920ec8bf98ad08e747c6
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    2e245e4 View commit details
    Browse the repository at this point in the history
  17. Copy kernel module source including subfolders

    Previously the build used the GN copy() target to copy the source
    of a to-be-built kernel module into the output directory. However,
    the target does not support preserving subfolder structure. Replace
    it with a Python script.
    
    A new 'source_dir_copy' template is added, which creates a dependency
    on the files in the source directory using the 'source_dir' target.
    Then it copies the files into 'target_out_dir'. When the source
    directory changes, the copying Python script is invoked again, updating
    the copy in the out folder.
    
    Change-Id: I23f2807fbb77bcf6d933e9541f5dbf1f2429b410
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    08b4869 View commit details
    Browse the repository at this point in the history
  18. Add Linux Hafnium module socket unit test

    Add a Linux module unit test that covers more cases than the basic load/unload.
    This test create a secondary VM, and using the socket interface sends a message
    and received the same payload as a response.
    
    Bug: 138977432
    Change-Id: Ibbc313c1a788924b2f227a28bf0ecd3bf21304a1
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    8762d5a View commit details
    Browse the repository at this point in the history
  19. Tidy up some code

    Bug: 138977432
    Change-Id: Ib3a826b4ea64b42cc229b81cd956f5ecb775d589
    Fuad Tabba authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    faa98b2 View commit details
    Browse the repository at this point in the history
  20. Refactor aarch64 barriers and TLBI commands

    Use macros instead of function calls to not rely on LTO inlining.
    Make macros take the op-kind argument to generalize.
    
    Test: ./kokoro/ubuntu/build.sh
    Change-Id: I8a5553d47cf3a0965fbf35d93c3c925f5f02ac4e
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    7ee68b7 View commit details
    Browse the repository at this point in the history
  21. Add missing instruction barriers to TLB flush code

    Also add comments explaining why individual barriers are needed.
    
    Change-Id: I2510ca56682129ffd754e09074e25c478b490c56
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    90a5af4 View commit details
    Browse the repository at this point in the history
  22. Flush all TLB entries if range too large

    Revisions of the ISA prior to ARMv8.4 do not support invalidation of
    a range of addresses, only looping per page or invalidating all. Hafnium
    currently only supports the former, this patch adds the latter for
    performance reasons.
    
    Change-Id: I2d07c66eaea0dabfdec1b63673103c1f43866784
    David Brazdil authored and efenniht committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    52fc202 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2020

  1. Enable MMU and caching in VM API tests

    VM API tests are failing on real hardware because VMs are not seeing
    data written by the hypervisor. The reason for this is that Hafnium has
    data caching enabled while the test VMs do not. Solve this discrepancy
    by enabling data caching in the VMs too, which requires enabling stage-1
    MMU translation.
    
    The entire address space is identity-mapped with read-write-execute
    permisssions. Only GIC tests currently require custom device mappings.
    
    Implementation shares ptable management code from src/mm.c and
    src/arch/mm.c.
    
    Bug: 138985026
    Test: ./kokoro/ubuntu/build.sh
    Change-Id: Ib9f599c448d70296a6ca869ddbb51abfcc55148d
    David Brazdil authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    b69a738 View commit details
    Browse the repository at this point in the history
  2. Make all TLB flushes apply to inner-shareable PEs

    Change-Id: I6e61d3a7e1cdf64f18119e946758923cec9764fb
    David Brazdil authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    d1b4760 View commit details
    Browse the repository at this point in the history
  3. Separate mm initialization from enablement.

    The configuration is static so it doesn't matter where it is originally
    calculated. There were previously multiple paths for enablement, one
    being implicit in initialization, so this splits them apart.
    
    Bug: 139269163
    Change-Id: Ic82c351b6e84b7d46e79f5886b39d07e53e6fde6
    AndrewScull authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    4d658ca View commit details
    Browse the repository at this point in the history
  4. Separate mm initialization from enablement for VMs.

    Change-Id: I891911bcd9087f584aed086f37b1aadf40673ce5
    AndrewScull authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    2fc5773 View commit details
    Browse the repository at this point in the history
  5. Save/restore mdcr_el2 with a vCPU system registers

    Monitor Debug Configuration Register (EL2) configures performance monitor
    extensions, which include debug as well as performance registers.  Trapping
    on certain accesses to these registers will likely vary between VMs.
    
    Make the code for saving/restoring register state when handling exceptions more
    flexible.  ldp/stp indices must be in the range [-512, 504].  Use post-index
    addressing to keep the base index register up to date.
    
    Bug: 132422368
    
    Change-Id: I5d095b19d8753f2a23b3b089f62a69041314ca08
    Fuad Tabba authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    a6464a9 View commit details
    Browse the repository at this point in the history
  6. Assign correct stack to CPUs.

    The CPU should get the corresponding entry from the callstacks for
    easier understanding. In particular, the boot CPU must get the 0th
    callstack since it has already been using that stack since boot.
    
    Change-Id: I435f5f9580baecae4be233507b72faa4da00d78f
    AndrewScull authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    3f54b1f View commit details
    Browse the repository at this point in the history
  7. Move barriers.h to aarch64 inc folder

    The file now contains aarch64 assembly and should not be in the
    architecture-agnostic folder.
    
    Change-Id: I4c5c7d98ffe75d95ee35fc360c025a8c68c10cb3
    David Brazdil authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    771978a View commit details
    Browse the repository at this point in the history
  8. Lock vCPU when accessing enabled_and_pending_count.

    Change-Id: Ic71d89c3b7a8acec71ac593228219e4044e58664
    AndrewScull authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    0cfe4ac View commit details
    Browse the repository at this point in the history
  9. Print all characters from debug log.

    Change-Id: I042384bbd33bcda3a8bde86f6af2beff23ce2d1a
    AndrewScull authored and efenniht committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    93d0ba8 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2020

  1. Introduce a DT-based manifest

    These are first steps towards a new manifest format. A new "device_tree"
    build target is introduced to compile DTS files to DTB, and
    `generate_initrd.py` now does not produce a "vms.txt" file. Instead
    "initrd" targets are expected to provide a path to a DTS manifest in the
    format:
    
        /dts-v1/;
    
        / {
          hypervisor {
            vm1 {
    	  debug_name = "primary";
    	};
    
    	vm2 {
    	  debug_name = "secondary1";
    	  kernel_filename = "filename";
    	  vcpu_count = <N>;
              mem_size = <M>;
    	};
    
    	...
          };
        };
    
    The information provided in the manifest matches "vms.txt".
    
    Bug: 117551352
    Test: manifest_test.cc
    Test: used by hftest
    Change-Id: I6b70bd44d2b110c4f7a6b971018c834084b6d8c4
    David Brazdil authored and efenniht committed Jan 29, 2020
    Configuration menu
    Copy the full SHA
    4c1ac8b View commit details
    Browse the repository at this point in the history
  2. Trap access to ICC_SRE_EL1 as well.

    Bug: 132960440
    Change-Id: I3e81ee7bc138f1c2fb411ec6b429b41b9ff630e5
    qwandor authored and efenniht committed Jan 29, 2020
    Configuration menu
    Copy the full SHA
    f48ebc3 View commit details
    Browse the repository at this point in the history
  3. std.c: Refactor, bring semantics closer to spec.

    Small refactor replacing the pattern of verifying inputs in
    safe equivalents of stdlib functions with a CHECK-like macro
    which adheres to the C11 semantics of filling the destination
    buffer when a constraint is not satisfied at runtime.
    
    Semantic changes:
    
    memcpy_s: More permissive memory ranges. Used to check that source does
    not overlap with the entire destination buffer, when only the first
    `count` bytes matter.
    
    memcpy_s: Used to allow `dest==src`. Safe under our implementation in
    aarch64 but not allowed under C11.
    
    strlen_s: Should return 'strsz' if NULL character not found.
    
    Change-Id: If483a97e6ee1c64c7f2afed9a0af1d3087da7002
    David Brazdil authored and efenniht committed Jan 29, 2020
    Configuration menu
    Copy the full SHA
    4441ef2 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2020

  1. Configuration menu
    Copy the full SHA
    21d3013 View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2020

  1. Configuration menu
    Copy the full SHA
    54c04da View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    37511a4 View commit details
    Browse the repository at this point in the history