Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Integrate with google/oss-fuzz for continuous fuzz testing? #1538

Open
nathaniel-brough opened this issue Jan 8, 2024 · 9 comments

Comments

@nathaniel-brough
Copy link
Contributor

Hey yoga team,

I've recently become interested in yoga. I'd like to suggest and champion an effort to set up some basic fuzz-testing and combine it with google/oss-fuzz for continuous fuzzing. I'm fully aware that you are very busy people and I don't want to overload your review/maintenance capacity. Is this a bad time to discuss potential security/reliability improvements?

If you're not familiar with fuzzing or oss-fuzz I've included a few brief notes below.

Benefits of Fuzz-Testing

  • Dynamic Code Testing: Fuzz-testing challenges systems with unexpected data, aiming to identify vulnerabilities or bugs. It’s akin to an exhaustive stress-test for the code.
  • Detecting Hidden Vulnerabilities: It can uncover potential weaknesses that may not be evident in routine tests.
  • Continuous and Automated Testing: With tools like Google’s OSS-Fuzz, fuzz-testing can be automated, running continuously on distributed systems, ensuring daily resilience checks.

Google/oss-fuzz for Continuous Fuzzing

  • Automated Fuzzing: OSS-Fuzz undertakes comprehensive fuzz-testing daily on a distributed cluster.
  • Detailed Reporting: OSS-Fuzz offers exhaustive reports in case of detected anomalies, enabling effective action.

I’d be more than happy to lead the effort in integrating fuzz testing with the yoga and assist in any way required.

Prior integrations

There have been a number of previous integrations completed with facebook repositories and google/oss-fuzz including;

  • facebook/time
  • facebook/zstd
  • facebookexperimental/starlark-rust (this was me)
  • facebook/proxygen
  • facebook/hermes
  • facebook/rocksdb

As a proof of concept I created a couple of super simple fuzz harnesses in #1537.

NOTE: Adding fuzz-testing and integrating with google/oss-fuzz was previously suggested here #1055 and was rejected. I think I've addressed the concerns raised in the first PR. While the original PR contained what was probably a higher performance fuzzer, the new fuzzer should be easier to integrate and doesn't introduce multiple sources of truth.

@NickGerleman
Copy link
Contributor

Thank you for raising this, and contributing a PR implementing it. I think it is a good idea, so long as we make sure it isn't disrupted by any unrelated Yoga changes.

@nathaniel-brough
Copy link
Contributor Author

As a follow up I'm going to start the integration with google/oss-fuzz if you are still keen on it. There is an application/integration process which I can complete on your behalf. To do this I'll need a couple of things from you;

  • A google/gmail email address that I can use for the primary contact. As far as I'm aware meta/facebook emails meet this requirement.
  • You to comment on the PR that I'll create that will form the integration/application saying that you approve of the integration.

Given the popularity of yoga I'm reasonably confident that it'll be accepted into oss-fuzz, but there is a non-zero chance that it could be rejected.

facebook-github-bot pushed a commit that referenced this issue Jan 19, 2024
Summary:
> This PR is based on the proposal in #1538 inlined below, to integrate yoga with `oss-fuzz`

Hey yoga team,

I've recently become interested in yoga. I'd like to suggest and champion an effort to set up some basic fuzz-testing and combine it with google/oss-fuzz for continuous fuzzing. I'm fully aware that you are very busy people and I don't want to overload your review/maintenance capacity. Is this a bad time to discuss potential security/reliability improvements?

If you're not familiar with fuzzing or oss-fuzz I've included a few brief notes below.

#### **Benefits of Fuzz-Testing**

- **Dynamic Code Testing**: Fuzz-testing challenges systems with unexpected data, aiming to identify vulnerabilities or bugs. It’s akin to an exhaustive stress-test for the code.
- **Detecting Hidden Vulnerabilities**: It can uncover potential weaknesses that may not be evident in routine tests.
- **Continuous and Automated Testing**: With tools like Google’s OSS-Fuzz, fuzz-testing can be automated, running continuously on distributed systems, ensuring daily resilience checks.

#### **Google/oss-fuzz for Continuous Fuzzing**

- **Automated Fuzzing**: OSS-Fuzz undertakes comprehensive fuzz-testing daily on a distributed cluster.
- **Detailed Reporting**: OSS-Fuzz offers exhaustive reports in case of detected anomalies, enabling effective action.

I’d be more than happy to lead the effort in integrating fuzz testing with the yoga and assist in any way required.

#### Prior integrations
There have been a number of previous integrations completed with facebook repositories and google/oss-fuzz including;
- facebook/time
- facebook/zstd
- facebookexperimental/starlark-rust (this was me)
- facebook/proxygen
- facebook/hermes
- facebook/rocksdb

As a proof of concept I created a couple of super simple fuzz harnesses in #1537.

NOTE: Adding fuzz-testing and integrating with google/oss-fuzz was previously suggested here #1055 and was rejected. I think I've addressed the concerns raised in the first PR. While the original PR contained what was probably a higher performance fuzzer, the new fuzzer should be easier to integrate and doesn't introduce multiple sources of truth.

Pull Request resolved: #1537

Reviewed By: yungsters

Differential Revision: D52800366

Pulled By: NickGerleman

fbshipit-source-id: 4957282456f3263e600d13ae6f3e983681bebda6
@nathaniel-brough
Copy link
Contributor Author

So I've got a draft PR ready to go integrating with OSS-fuzz in google/oss-fuzz#11533. The only thing I need to do now is add a primary contact. I've added [email protected] which seems to be CC'd on most of the other facebook integrations.

@NickGerleman
Copy link
Contributor

Thanks @silvergasp. Is there anything left to action on from our side?

@nathaniel-brough
Copy link
Contributor Author

Yeah, all I need is the email address that you'd like to use to get access to the bug tracker and oss fuzz dashboard. Just as a note, it'll be stored as plain text on the oss fuzz git repo. After that everything should be good to go.

@NickGerleman
Copy link
Contributor

Reusing oss-fuzz probably makes sense if it is already being used, though I’m not actually sure where it is funelling to. I can try to find that out.

@nathaniel-brough
Copy link
Contributor Author

Easy do you want me to add your email to the config? That will give you personal access to the dashboard/bug tracker?

@NickGerleman
Copy link
Contributor

I appreciate it. Could you use this address? https://github.com/facebook/yoga/commit/cc2a87d4f95d2deac3f4ffed02ca4c130aec70ba.patch

@nathaniel-brough
Copy link
Contributor Author

Ah sorry I missed this notification. I've pulled the trigger on the PR over at oss-fuzz now. All that is needed is to wait for the oss-fuzz team to approve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants
@NickGerleman @nathaniel-brough and others