Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose exceptions in DumpProcessingController #526

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Expose exceptions in DumpProcessingController #526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
be1f1af
throw exceptions when they occur
brett-matson Jan 19, 2021
1a1299a
declare that example main methods throw IOException
brett-matson Jan 19, 2021
b38cd5e
Merge branch 'master' into brett-matson-exceptions
wetneb Mar 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions wdtk-dumpfiles/src/main/java/org/wikidata/wdtk/dumpfiles/DumpProcessingController.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,8 @@ public Sites getSitesInformation() throws IOException {
* @see DumpProcessingController#processDump(MwDumpFile)
* @see DumpProcessingController#getMostRecentDump(DumpContentType)
*/
public void processAllRecentRevisionDumps() {
public void processAllRecentRevisionDumps()
throws IOException, FileAlreadyExistsException {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FileAlreadyExistsException is a sub class of IOException (same in other places)

Copy link
Member Author

@wetneb wetneb Mar 24, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps there is still a case for including both of them explicitly in the signature, because the caller might want to handle them differently?

WmfDumpFileManager wmfDumpFileManager = getWmfDumpFileManager();
if (wmfDumpFileManager == null) {
return;
Expand All @@ -398,7 +399,8 @@ public void processAllRecentRevisionDumps() {
*
* @see DumpProcessingController#processAllRecentRevisionDumps()
*/
public void processMostRecentMainDump() {
public void processMostRecentMainDump()
throws IOException, FileAlreadyExistsException {
DumpContentType dumpContentType;
if (this.preferCurrent) {
dumpContentType = DumpContentType.CURRENT;
Expand All @@ -419,7 +421,8 @@ public void processMostRecentMainDump() {
*
* @see DumpProcessingController#processAllRecentRevisionDumps()
*/
public void processMostRecentJsonDump() {
public void processMostRecentJsonDump()
throws IOException, FileAlreadyExistsException {
processDump(getMostRecentDump(DumpContentType.JSON));
}

Expand All @@ -434,7 +437,8 @@ public void processMostRecentJsonDump() {
* @param dumpFile
* the dump to process
*/
public void processDump(MwDumpFile dumpFile) {
public void processDump(MwDumpFile dumpFile)
throws IOException, FileAlreadyExistsException {
if (dumpFile == null) {
return;
}
Expand Down Expand Up @@ -493,18 +497,23 @@ public MwDumpFile getMostRecentDump(DumpContentType dumpContentType) {
* the dump file processor to use
*/
void processDumpFile(MwDumpFile dumpFile,
MwDumpFileProcessor dumpFileProcessor) {
MwDumpFileProcessor dumpFileProcessor)
throws IOException, FileAlreadyExistsException {
try (InputStream inputStream = dumpFile.getDumpFileStream()) {
dumpFileProcessor.processDumpFileContents(inputStream, dumpFile);
} catch (FileAlreadyExistsException e) {
logger.error("Dump file "
String errorMessage = "Dump file "
+ dumpFile.toString()
+ " could not be processed since file "
+ e.getFile()
+ " already exists. Try deleting the file or dumpfile directory to attempt a new download.");
+ " already exists. Try deleting the file or dumpfile directory to attempt a new download.";
logger.error(errorMessage);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the logging is still useful now that we raise the exception again.

throw new FileAlreadyExistsException(errorMessage);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to give to the constructor the file names to allow getFile() to be used by the caller. C.f. doc

} catch (IOException e) {
logger.error("Dump file " + dumpFile.toString()
+ " could not be processed: " + e.toString());
String errorMessage = "Dump file " + dumpFile.toString()
+ " could not be processed: " + e.toString();
logger.error(errorMessage);
throw new IOException(errorMessage);
}
}

Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/ClassPropertyUsageAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ public int compare(
* which dump file to use and whether to run in offline mode, modify the
* settings in {@link ExampleHelpers}.
*/
public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
ClassPropertyUsageAnalyzer.printDocumentation();

Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/EntityStatisticsProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ static class UsageStatistics {
*
* @param args
*/
public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
EntityStatisticsProcessor.printDocumentation();

Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/ExampleHelpers.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ public static void configureLogging() {
* the object to use for processing entities in this dump
*/
public static void processEntitiesFromWikidataDump(
EntityDocumentProcessor entityDocumentProcessor) {
EntityDocumentProcessor entityDocumentProcessor) throws IOException {

// Controller object for processing dumps:
DumpProcessingController dumpProcessingController = new DumpProcessingController(
Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/GenderRatioProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ public int compare(SiteRecord o1, SiteRecord o2) {
* results to a file. To change which dump file to use and whether to run in
* offline mode, modify the settings in {@link ExampleHelpers}.
*/
public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
GenderRatioProcessor.printDocumentation();

Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/LifeExpectancyProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ public class LifeExpectancyProcessor implements EntityDocumentProcessor {
* offline mode, modify the settings in {@link ExampleHelpers}.
*
*/
public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
LifeExpectancyProcessor.printDocumentation();

Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/LocalDumpFileExample.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ public class LocalDumpFileExample {
*/
private final static String DUMP_FILE = "./src/resources/sample-dump-20150815.json.gz";

public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
LocalDumpFileExample.printDocumentation();

Expand Down
4 changes: 3 additions & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/TutorialExample.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
* #L%
*/

import java.io.IOException;

import org.wikidata.wdtk.datamodel.interfaces.EntityDocumentProcessor;
import org.wikidata.wdtk.dumpfiles.DumpProcessingController;
import org.wikidata.wdtk.dumpfiles.MwRevision;
Expand All @@ -41,7 +43,7 @@
*/
public class TutorialExample {

public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();

// Controller object for processing dumps:
Expand Down
2 changes: 1 addition & 1 deletion wdtk-examples/src/main/java/org/wikidata/wdtk/examples/WorldMapProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ public class WorldMapProcessor implements EntityDocumentProcessor {
*
* @param args
*/
public static void main(String[] args) {
public static void main(String[] args) throws IOException {
ExampleHelpers.configureLogging();
WorldMapProcessor.printDocumentation();

Expand Down