Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replacing Jackson Factory with Gson Factory #32148

Closed
wants to merge 2 commits into from
Closed

Replacing Jackson Factory with Gson Factory #32148

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
be64a84
[Task]: Replace JacksonFactory references with GsonFactory #32130
dragonHead1001001 Aug 11, 2024
dd3fa04
Replacing Jackson Factory with Gson Factory in Transport.java
dragonHead1001001 Aug 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions gradle/wrapper/gradle-wrapper.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#Sat Aug 10 17:06:24 IST 2024
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this was added by mistake, could you please remove it?

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip
Expand Down
4 changes: 2 additions & 2 deletions ...-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/Transport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the precommits (https://github.com/apache/beam/actions/runs/10342154630/job/28624760220?pr=32148), I see the following error:

Execution failed for task ':sdks:java:extensions:google-cloud-platform-core:analyzeClassesDependencies'.
> Dependency analysis found issues.
  usedUndeclaredArtifacts
   - com.google.http-client:google-http-client-gson:1.44.1@jar
  unusedDeclaredArtifacts
   - com.google.http-client:google-http-client-jackson2:1.44.1@jar

I think we need to replace the jackson artifact with the gson one here -

beam/sdks/java/extensions/google-cloud-platform-core/build.gradle

Line 42 in dd3fa04

implementation library.java.google_http_client_jackson2

(example where we've already included the gson library -

beam/sdks/java/io/solace/build.gradle

Line 50 in 780eef9

implementation library.java.jackson_core
)

import com.google.api.services.storage.Storage;
import com.google.auth.Credentials;
import com.google.auth.http.HttpCredentialsAdapter;
Expand Down Expand Up @@ -53,7 +53,7 @@ private static class SingletonHelper {

static {
try {
JSON_FACTORY = JacksonFactory.getDefaultInstance();
JSON_FACTORY = GsonFactory.getDefaultInstance();
HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
} catch (GeneralSecurityException | IOException e) {
throw new RuntimeException(e);
Expand Down
10 changes: 5 additions & 5 deletions ...loud-platform-core/src/test/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
import com.google.api.client.json.GenericJson;
import com.google.api.client.json.Json;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.testing.http.HttpTesting;
import com.google.api.client.testing.http.MockHttpTransport;
import com.google.api.client.testing.http.MockLowLevelHttpRequest;
Expand Down Expand Up @@ -474,7 +474,7 @@ public void testRetryFileSizeNonBatch() throws IOException {

@Test
public void testGetSizeBytesWhenFileNotFoundBatch() throws Exception {
JsonFactory jsonFactory = new JacksonFactory();
JsonFactory jsonFactory = new GsonFactory();

String contentBoundary = "batch_foobarbaz";
String contentBoundaryLine = "--" + contentBoundary;
Expand Down Expand Up @@ -545,7 +545,7 @@ public void testGetSizeBytesWhenFileNotFoundNoBatch() throws Exception {

@Test
public void testGetSizeBytesWhenFileNotFoundBatchRetry() throws Exception {
JsonFactory jsonFactory = new JacksonFactory();
JsonFactory jsonFactory = new GsonFactory();

String contentBoundary = "batch_foobarbaz";
String contentBoundaryLine = "--" + contentBoundary;
Expand Down Expand Up @@ -657,7 +657,7 @@ public LowLevelHttpResponse execute() throws IOException {

@Test
public void testRemoveWhenFileNotFound() throws Exception {
JsonFactory jsonFactory = new JacksonFactory();
JsonFactory jsonFactory = new GsonFactory();

String contentBoundary = "batch_foobarbaz";
String contentBoundaryLine = "--" + contentBoundary;
Expand Down Expand Up @@ -1045,7 +1045,7 @@ private void verifyMetricWasSet(
/** Builds a fake GoogleJsonResponseException for testing API error handling. */
private static GoogleJsonResponseException googleJsonResponseException(
final int status, final String reason, final String message) throws IOException {
final JsonFactory jsonFactory = new JacksonFactory();
final JsonFactory jsonFactory = new GsonFactory();
HttpTransport transport =
new MockHttpTransport() {
@Override
Expand Down
4 changes: 2 additions & 2 deletions ...a/org/apache/beam/sdk/extensions/gcp/util/LatencyRecordingHttpRequestInitializerTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
import com.google.api.client.http.LowLevelHttpRequest;
import com.google.api.client.http.LowLevelHttpResponse;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.storage.Storage;
import java.io.IOException;
import org.apache.beam.sdk.metrics.Histogram;
Expand All @@ -61,7 +61,7 @@ public class LatencyRecordingHttpRequestInitializerTest {
@Mock private LowLevelHttpResponse mockLowLevelResponse;
@Mock private Histogram mockHistogram;

private final JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
private final JsonFactory jsonFactory = GsonFactory.getDefaultInstance();
private Storage storage;

@Before
Expand Down
4 changes: 2 additions & 2 deletions ...rc/test/java/org/apache/beam/sdk/extensions/gcp/util/RetryHttpRequestInitializerTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
import com.google.api.client.http.LowLevelHttpRequest;
import com.google.api.client.http.LowLevelHttpResponse;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.testing.http.MockHttpTransport;
import com.google.api.client.testing.http.MockLowLevelHttpRequest;
import com.google.api.client.testing.http.MockLowLevelHttpResponse;
Expand Down Expand Up @@ -79,7 +79,7 @@ public class RetryHttpRequestInitializerTest {
@Mock private LowLevelHttpResponse mockLowLevelResponse;
@Mock private HttpResponseInterceptor mockHttpResponseInterceptor;

private final JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
private final JsonFactory jsonFactory = GsonFactory.getDefaultInstance();
private Storage storage;

// Used to test retrying a request more than the default 10 times.
Expand Down
3 changes: 1 addition & 2 deletions ...platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HttpHealthcareApiClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.healthcare.v1.CloudHealthcare;
import com.google.api.services.healthcare.v1.CloudHealthcare.Projects.Locations.Datasets.FhirStores.Fhir.PatientEverything;
import com.google.api.services.healthcare.v1.CloudHealthcare.Projects.Locations.Datasets.Hl7V2Stores.Messages;
Expand Down Expand Up @@ -736,7 +735,7 @@ private void initClient() throws IOException {

client =
new CloudHealthcare.Builder(
new NetHttpTransport(), new JacksonFactory(), requestInitializer)
new NetHttpTransport(), new GsonFactory(), requestInitializer)
.setApplicationName("apache-beam-hl7v2-io")
.build();
httpClient =
Expand Down
6 changes: 3 additions & 3 deletions ...-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImplTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
import com.google.api.client.http.LowLevelHttpResponse;
import com.google.api.client.json.GenericJson;
import com.google.api.client.json.Json;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.testing.http.MockHttpTransport;
import com.google.api.client.testing.http.MockLowLevelHttpRequest;
import com.google.api.client.testing.util.MockSleeper;
Expand Down Expand Up @@ -1597,12 +1597,12 @@ public void testSkipInvalidRowsIgnoreUnknownIgnoreInsertIdsValuesStreaming()
/** A helper to convert a string response back to a {@link GenericJson} subclass. */
private static <T extends GenericJson> T fromString(String content, Class<T> clazz)
throws IOException {
return JacksonFactory.getDefaultInstance().fromString(content, clazz);
return GsonFactory.getDefaultInstance().fromString(content, clazz);
}

/** A helper to wrap a {@link GenericJson} object in a content stream. */
private static InputStream toStream(GenericJson content) throws IOException {
return new ByteArrayInputStream(JacksonFactory.getDefaultInstance().toByteArray(content));
return new ByteArrayInputStream(GsonFactory.getDefaultInstance().toByteArray(content));
}

/** A helper that generates the error JSON payload that Google APIs produce. */
Expand Down
4 changes: 2 additions & 2 deletions ...le-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIOTestUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import com.google.api.client.http.HttpHeaders;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.healthcare.v1.model.HttpBody;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.model.StorageObject;
Expand Down Expand Up @@ -136,7 +136,7 @@ public static void tearDownTempBucket() throws IOException {
request.setReadTimeout(60000); // 1 minute read timeout
};
Storage storage =
new Storage.Builder(new NetHttpTransport(), new JacksonFactory(), requestInitializer)
new Storage.Builder(new NetHttpTransport(), new GsonFactory(), requestInitializer)
.build();
List<StorageObject> blobs = storage.objects().list(DEFAULT_TEMP_BUCKET).execute().getItems();
if (blobs != null) {
Expand Down
Loading