forked from Nandaka/PixivUtil2
-
Notifications
You must be signed in to change notification settings - Fork 0
/
changelog.txt
1276 lines (1072 loc) · 52.1 KB
/
changelog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
20230105
- Fix #1219: ai filtering logic for tag search.
- Fix cacert resolution #1220 and remove ansi code from logging
20230102
- Implement #1191: add wait between search API
- Implement #1187: Handle OSErrors in a cleaner way
- Implement #1176: Failed ugoira conversions counted as successful downloads
- Implement #439: Have options to download both R-18/18G or one over other
Add new config `r18Type` with value `0` = both R18 and R-18G, `1` = only R18, or `2` = only R18G
- Merge fixs socks5 `getaddrinfo`&dns leak, and enable auth (#1195) by @NewUserHa
- Merge Optimize visual quality of converted ugoira (#1198) by @NewUserHa
- Merge Enhance document (#1197) by @NewUserHa
- Merge fix: infofile add extension name first (#1203) by @神楽坂みずき
- temp fix? #1209, you also need to change the user agent to 'Mozilla/5.0' only in config.ini
- Merge get rid of useless checks to speed up the download, if image not exists at all. (#1201) by @NewUserHa
Also Fix #1206: Update retry logic for incomplete download.
- Merge Add the unused configuration option "extensionFilter" to the app. (#1217) by @Terryble2
- Merge Don't process member page if it's in blacklist_member.txt (#1208) by @NewUserHa
- Merge Update readme.md (#1202) by @NewUserHa
- Merge Add mkv ugoira packing without encoding (#1218) by @PatrickL546
- Merge Update to python 3.11 for faster startup for docker (#1199) by @NewUserHa
- Implement #1216, set useTranslatedTag = True in config.ini to enable
- Merge fix: remove trim `_pxx` (#1204) by @ミズキ
also add logic to only trim p_xx for image info when image mode == manga / multi images
- Implement #1189, set `aiDisplayFewer=True` in config.ini to enable.
- Add configurable download buffer in config.ini
- Add color to ui for download by image and by member.
20221029
- Merge Fix issue with fanbox coverImageUrl (#1181) (#1185) by @KamenReader
- Update #1184: fix .gitignore for venv.
20220924
- Update #1156: add support for url provider 'fanbox.creator'.
- Merge bookmark skip function improvement (#1160) by @Yuki.
- Implement #1162: add disableScreenClear in config.ini in debug section,
- Implement #1165: Skip retry if the archive file format not supported.
- Update #1167: change default downloadDelay to 5s.
- Update #1116: update readme.md
- Increase download buffer size from 8kB to 128kB.
- update compiled version to use Python 3.10.7 64-bit.
20220825
- Fix #1142 and #1152: Fanbox Service Provider error.
- Update #814: support to import full cookies string.
- Merge fix ugoira generation using ffmpeg.exe Windows path for all distribution (#1145) by @Toyem
- need to update `config.ini` in `[FFmpeg]` section if you have parameter error when encoding to gif.
`gifParam = -filter_complex [0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle`
- Fix #1146: update lxml with html5lib.
- Merge Add date option to ranking download (#1149) by @Toyem
20220804
- Merge fix reencoding backup of files to follow ugoira config (#1131) by @Toyem.
- Implement #1133: enhance content_provider.json to support urlEmbed.
- Merge Add a separator in temp folder cmd for windows (fix #1132) (#1135) by @Toyem.
- Merge fix ugoira generation and reencoding for all distribution for issue #1132 (#1136) by @Toyem.
- Merge Fix for #1138 (#1139) by @NHOrus.
- Merge Fixes Byte Order Mark in Downloaded file list, again. (#1137) by @NHOrus.
- and other minor changes.
20220701
- Merge Update WriteXMP feature to use pyexiv2 only (#1043) by @Baa.
- Merge Adds feature to strip HTML from captions (#1045) by @Baa.
- Fix #1047: fix "ffmpegExt" config.
- Fix #1019: add 404 handling for pixiv sketch
- Merge Some optimizations (#1052) by @amatuerCoder.
- Implement #1064: add parser for titleCaptionTranslation.
- Add new filename format %translated_title% for translated title if exists, else it will use original title.
- Implement #1068: add content type filter.
- Merge Fix #1072, #1065 (running in pipe on Unix) (#1082) by @dragontamer8740.
- Fix #1084: Fanbox download removes Pixiv files with same ID.
- Implement #1087: FANBOX's url_embed not detected.
- Update logging for #1090: premature download by tags.
- Merge Pixiv Sketch config and Database Manager changes (#1078) by @cglmrfreeman.
- Fix #1094: Fanbox f1/f2 download issue.
- Fix #1095: Fanbox Post missing files to download.
- Merge ugoira re-encode and export local database option (#1113) by @Toyem.
- Merge Fix option name collision (#1114) by TwinkleStar03.
- Merge Code improvement based on @PatrickL546 's script (#1112) (#1115) by @Toyem.
- Implement #1117: add new filename format %fanbox_name% for fanbox specific artist name.
- Merge Update readme (#1121) by @PatrickL546.
- Merge Update on the reencoding function (#1123) by @Toyem.
- Fix #1126 use ffmpeg from config.ini
- Merge Fixes my bad on past attempt to deal with BOM (#1130) by @NHOrus.
- Implement #1128
- add new configuration `disableLog` in `config.ini` in `Debug` section. Set to `True` to disable logging.
20211104
- Fix Issue #1036: update error message for ffmpeg and add option to enable verboseOutput from ffmpeg.
- remove `-lossless 1` from webm param due to ffmpeg memory issue might cause ugoira not fully converted.
- remove logging from ffmpeg output due to conflict with logger
- Update ffmpeg detection to return warning if ffmpeg return exit code != 0 when loading for the first time.
- Update default encoding parameter for webm and webp
- ffmpegParam = -row-mt 1 -deadline good -crf 20 -vsync 2 -r 999 -pix_fmt yuv420p
- webpParam = -row-mt 1 -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999
20211030
- Fix Issue #1035: enhance error detection when converting ugoira.
- Fix Issue #1031 and #1034: fix command line option for 3 and 8
20211020
- Fix Issue #1028: Update download new illust parser to use new API.
- Implement #1027: Add Bookmark Count Option to Option 5, 8.
- Merge Update PixivFanboxHandler.py (#1026) by amatuerCoder.
- Add new option for customCleanUpRe in config.ini.
- Keep case when writing config.ini.
- Update #1004 rewrite pyexiv2 to version 2.7.0.
- Merge fix(deps): demjson (#1015) by BassOfBass.
- Fix #1011: handle user not exists during download.
- Fix check if pixiv sketch return empty post #1013 #1014.
- Implement #1005: Resolve and download Pixiv by Fanbox "artist/creator" id.
- Merge Correct shape of BOM in Downloaded List (#1010) by NHOrus.
- Implement #1007: Add option to download Pixiv Sketch using option 4.
- Add check for R-18/R-18G from pixiv website setting.
- Implement #1002: update url dump for fanbox and add some typing.
20210822
- Merge Fix #993 (#995) by amatuerCoder.
- Merge Get member information adjustment (#992) by amatuerCoder.
- Merge Fixes some bugs with XMP sidecars. (#985) by Baa.
- Merge Adds support for .XMP sidecar files. (#980) by Baa.
- Merge Allows Post Processing to accept command arguments (#979) by Baa.
20210702
- Update python to 3.9.6 and latest libraries.
- Skip processing members at all if image list is empty (no folder/avatar/bg will be processed).
- Merge fixes from amatuerCoder.
- #964, #963, #954, #945, #951
- Merge Change shebang to python3 (#958) by KLsz.
- Implement #956 additional filename format: %md5%, %sha1%, and %sha256%.
Please note that the file-exists check might not works due to the hash can only be calculated after the file is downloaded.
- Merge fixed an ANSI escape code (#944) by DenDen047.
- Merge Fix argument mismatch in process_manga_series (#939) by dddlr.
- Merge fixed regression in console title: using XTerm OSC (#950) by whinette.
- Update #930, replace resized cover with full size instead.
- Fix Issue #942: AttributeError when picking "e. Export online bookmark"
- Implement #940: add new name format: %force_extension{.*}% to force specific extension.
Usage = %force_extension{jpeg}% to always set the extension to .jpeg
- Implement #970: add feature to perform post processing for downloaded files.
Add config in [DownloadControl] section:
- enablePostProcessing = default False. Set to true to enabled post processing cmd for every downloaded files
- postProcessingCmd = command to execute. add %filename% to pass the downloaded filename.
NO ERROR HANDLING AT ALL, use on your own risk.
20210421
- Implement #930: download full size image for fanbox cover.
- Fix #929: update fanbox post parser for post type = entry.
- Merge Show URL for oauth (#931) by Jhyrachy.
20210326
- Implement #706: add novel support.
- Merge Update PixivDBManager.py (#910) by amatuerCoder.
- Merge Update PixivBrowserFactory.py (#908) by amatuerCoder.
- Merge Fix #889 #899 (#903) by amatuerCoder.
- Update test script for PixivBrowserFactory.
- Implement #928: allow to use image tags for filtering when downloading user bookmark (option 6).
- Merge Add config option to ignore beta versions (#923) by Terryble2
- Merge Bad chars enhancement (#920) by amatuerCoder.
- Update error message for #914
20210128
- Fix Issue #878: update search by tag parser to handle missing 'isAdContainer'.
- Fix Issue #877: update fanbox parser for missing embedId.
- Fix Issue #881: change content_provider.json path resolution.
- Partial Implement #873: store member token in db.
- Fix Issue #880: update batch handler based on changes from byjtje.
- Change from re.match() to re.search() for extension filter for batch job.
- Fix Issue #882: trying to load from None object.
- Implement #891: add new filename format '%bookmarks_group%', refer to https://dic.pixiv.net/a/users%E5%85%A5%E3%82%8A for grouping rule.
- Implement #885: allow to configure the parameter to convert gif and apng.
- Fix Issue #893: fix useTagsAsDir logic.
- Fix Issue #892: PixivSketch now load all images, including R-18 (follow your settings in the website).
- Merge Fix #873 and #886 (#888) by amatuerCoder.
- Fix Issue #883: update parser for avatar image.
- Merge Remove double slash (#894) by Yuki.
- Merge Fix (#897) by amatuerCoder.
- Update cloudscraper to 1.2.54
20201130
- Merge Added series JSON and an option to download it, expended %manga_series% to work with all modes, added an option for untouched JSON (#855) by byjtje.
- Merge fix bad standard config value (#857) by byjtje.
- Fix Issue #860: follow numberofpage in config.ini
- Merge Add config option to not download the cover for fanbox (#865) by qazmlpok.
- Merge Allow pasting of urls for input (#867) by byjtje.
- Update #867 to handle pixiv sketch and fanbox username with symbol.
- other small fixes.
20201031
- Merge Fix bug of header (#853) by markhuang3310.
- Merge Added Series Info and conditionals to JSON/txt, and removed an unneccesary string format (#851) by byjtje.
- Update readme.md (please read...)
- monkey-patch cloudscraper method for browsers.json filepath resolution.
- update cacert.pem path resolution to count if the exe being called from different folder
20201025
- Merge To be consistent with PixivBatchHandler. Fix #844 (#845) by fireattack.
- Implement #842: to use cacert.pem from the root directory if frozen.
- need to modify cloudscraper library to inherit 'verify' attribute from given Session object
- Merge Sketch default date fix (#841) by byjtje
20201024
- add cmd logging when converting ugoira.
- Implement #406: add option 13: download by manga series
- Specific for filenameMangaFormat in Manga Series mode (option 13):
-> %manga_series_order%
the order in the manga series.
-> %manga_series_id%
original manga series id.
-> %manga_series_title%
original manga series title, different from work title.
-> %manga_series_desc%
original manga series description, different from work description.
- Merge #806 fix for Fanbox EndPage input parameter (#808) by amatuerCoder.
- Implement #812: handle different Sketch and Pixiv artist id.
- add new filename format `%sketch_member_id%` for PixivSketch
- Fix Issue #822: Update parser for user's image bookmark.
- Fix Issue #827: Update parser to NOT use AppAPI/oAuth to get member token.
- Fix Issue #824: Update argument parser for option 6.
- Add new configuration `checkLastModified` in `[DownloadControl]` for Issue #807
- This will compare last-modified time with works date, require `setlastmodified = True` in config.ini to work properly.
- Throw error if cannot get the user id during login, related to #831
- Merge Adding Filename option for Sketch and fixing dates and Unix filenames (#836) by byjtje
- change default dateFormat to %Y-%m-%d (YYYY-MM-DD)
- Fix Issue #838: delete zip/ugoira handling.
- Add new start parameter:
- --start_date and --end_date for option 3, 7, and 9
- --tag_sort_order to set sorting order for option 3 and 7
- Add check if ffmpeg exist or not if ugoira conversion is enabled.
- Implement #837: add new config to skip downloading if `alwaysCheckFileSize = True` and remote file size is unknown => skipUnknownSize.
- Update #814: replace requests with cloudscrapper for oAuth handling.
- Implement #834: additional tag sorting order for premium users.
- order=popular_d (overall), popular_male_d (popular among male users) and popular_female_d.
- Merge Fix error in PixivHelper.py:173 - TypeError expects String, not Int (#832) by Jared Shields.
- update test cases and libraries.
20201006
- Implement #797: reload blacklist on after reloading main menu.
- Implement #793: allow to download sketch when using option 1.
- Update #796: change apng converter to use ffmpeg.
- Fix Issue #795: handle no avatar/bg.
- Implement #802: use ffmpeg for converting to gif.
- Merge Implemented #804 download FANBOX from list file (#805) by amatuerCoder
- Added new option `listPathFanbox`
- IMPORTANT: changes the command line parameters, run with --help for more details
- Update readme.md
20200915
- Fix Issue #778: "'list' object has no attribute 'startswith'" process_tags_list().
- Update #777: add s_tag and date_d param value
- Fix Issue #772: Pixiv Sketch tz_info
- Update #768: enhance the error message
- Implement #726: file extension filter for batch job
- Merge: Disable blacklist for the download by image id option (#791) by byjtje.
- Merge: Update PixivBrowserFactory.py (#790) by amatuerCoder.
- Merge: Fixed avatar/background default file extension and background urlfilename (#789) by byjtje.
- Implement #782: add filename in the warning message.
- Merge: Removed unneccessary check from check_version and made the browser opening opt-in (#779) by byjtje.
20200824
- Implement #650: open default browser to release page if checkNewVersion=True in config.ini
- Merge: Add run on repl.it badge to README (#770) by cyleja1234
- Merge: Added date update for artists with no pictures (#766) by byjtje
- Merge: Switched the check from last image to last downloaded to avoid always checking when it doesn't download any images (#764) by byjtje
- Merge: Minor changes to PixivFanboxHandler, PixivUtil2, readme (#763) by amatuerCoder
- Merge: Regex Title Filtering + Date updates and a few minor changes (#759) by byjtje
- Merge: Update readme.md (#754) by amatuerCoder
- Merge: proper checkUpdatedLimit check (#753) by byjtje
- Implement #749: Pixiv Sketch support.
20200726
- Update #592: dns over socks proxy handling.
- Merge: Fix shebang of PixivUtil2.py (#739) by RoestVrijStaal
- Merge: Dockerfile (#740) by DenDen047
- Merge: Fixed handle tags (#741) by byjtje
- Fix Issue #743: add error message in config item, add colorama.
- Implement #718: add ffmpegExt in config.ini to control the file extension.
- Merge: Update Fanbox Handling (#746) by amatuerCoder
- Refactor process FANBOX to PixivFanboxHandler
- Add update check for refresh_token for writing config.ini
- Merge: Update PixivFanboxHandler.py (#747) by amatuerCoder
20200712
- Merge Added a filter for Titles and added a condition to prevent unnecessary blacklist checks (#737) by byjtje.
- Implement #716: include soundcloud provider support.
- Implement : add joboption for r18mode for batch.
- Implement #719: update message for menu_download_from_online_image_bookmark().
- Fanbox related fixes by amatuerCoder.
- update readme.md.
20200524
- Fix #697 bookmark filtering for premium user
- Fix #713 allow %search_tag% format in option 6
- Merged HTML style adjustment for FANBOX (#711) by amatuerCoder.
20200502
- Merged FANBOX download enhancement (#680) by amatuerCoder.
- Merged Optimize Query `insertMangaImage` & Typo Fix (#678) by Jared Shields.
- Merged Simplify(?) codes with reflection(?) thoughts (#676) by amatuerCoder.
- Fix Issue #684: cannot use pixiv and fanbox cookie together.
- Fix Issue #687: merged When artist is None (#683) by amatuerCoder.
- Merged Indent/print-out adjustment & new method to update FANBOX cookie (#694) by amatuerCoder.
- Implement #197: new option 'b' to download from batch_job.json
See https://github.com/Nandaka/PixivUtil2/wiki/Using-Batch-Job-(Experimental) for the sample template.
- Refactor out codes for handling batch job.
- Fix Issue #696: wrong import for PixivException
- Merged FANBOX database patch (#695) by amatuerCoder.
202000424
- Try to fix #658, #662: login detection.
- Try to fix #654: explicitly call close() when writing config.ini.tmp
- Implement #652: Enable filter by bookmark count for Pixiv Premium.
- Merged #656 (#659) by amatuerCoder: Download fanbox articles in HTML.
- set writeHtml = True to enable.
- modifieable template based on pattern.html
- Merged #660 by wmjdgla: Simplified print_progress and fixed jerky progress bar for indeterminate case.
- Merged #647 by Alex: Refactor sanitize_filename in PixivHelper.
- Merged #671 by amatuerCoder: Html improvements for Fanbox.
- Merged #670 by Jared Shields: Update autoAddMember.
- Fix Issue #663: handle directory name for filename formats.
- Merged #661 by amatuerCoder: Added logic for links if links token was in p blocks
- Update test cases and other small fixes (see the commit history for more details).
20200307
- Fix Issue #648: Can't download Ugoiras when "downloadresized" is active.
20200228
- Fix Issue #641: update API endpoints for Pixiv FANBOX.
- Implement #639: add option to download the resized image, rather than original size.
set `downloadResized = True` at `[DownloadControl]` section in config.ini to enable.
20200125 - Happy CNY 2020!
- Update #623: change default local to '', if exists then use the current locale.
- Update test cases.
- Implement #625: Add option to automatically add downloaded member to DB.
Set 'autoaddmember = True' in config.ini to enable.
- Implement #526: Add menu to import list.txt on demand.
- Implement #575: Add option for filename format for manga's info.
Configure 'filenamemangainfoformat' in config.ini to enable.
- Update #622: use new API endpoint for option 5.
- Fix Issue #627: sanizite trailing '.' in the filename.
- Fix Issue #626: remove redundant message for option 4.
- Update #611: avoid duplicate file for Pixiv FANBOX.
20200112
- Fix Issue #611: updated FANBOX post parser to support old post.
- Fix Issue #612: moved wait statement due to #609 changes.
- Fix Issue #613: updated logic check due to #609 changes.
- Merge #615 and #617: updated utf8 supported FS check on linux.
- Fix Issue #618: update requirements.txt, replace socksipy-branch to PySocks.
- Fix Issue #622, #623, and Merge #624: update bookmark and image bookmark handler.
20200101 - Happy New Year!
- Fix Issue #607: fix db path for parseBookmark call.
- Fix Issue #608: handle user suspended for Fanbox.
- Implement #567: add minFileSize and maxFileSize in config.ini, set 0 to disable.
- Reorder some settings in config.ini to new section [DownloadControl]
- overwrite = False
- backupOldFile = False
- dayLastUpdated = 7
- alwaysCheckFileSize = False
- checkUpdatedLimit = 0
- useBlacklistTags = False
- dateDiff = 0
- enableInfiniteLoop = False
- useBlacklistMembers = False
- Implement #609: separate 'checkUpdatedLimit' and 'alwaysCheckFileSize' validation.
- Fix Issue #606: set encoding to utf8.
- Implement #595: implement SSL bypass for most connection.
20191221
- Fix Issue #600: update for login detection.
20191218
- Upgrade to python 3.8.0
- Fix Issue #601: update bookmark image parser.
- Fix Issue #593: handle memory leak, require python 3.8.0
- Implement #597: update fanbox parser to ensure image is saved on order by amatuerCoder.
- Update test cases.
20191215
- Fix Issue #594: download by tag.
- Fix Issue #592: update socks proxy handler.
- Fix Issue #590: update webm creation handler.
- Fix Issue #591: update apng creation handler.
20191214
- Moved to python 3.
- Fix Issue #580: update error handler for suspended user.
- Fix Issue #579: update remote filesize parser.
- Fix Issue #586: update content_provider.json for vimeo.
20191118
- Merged #571: fix password check logic.
- Merged #572: fix Issue #565 with negative time zone offset.
- Implement #569: Add delay for downloading bookmark pages.
20191116
- Fix Issue #567: add try-catch for suspended user for FANBOX.
- Fix Issue #568: update post counter limit to 60.
20191111
- Fix Issue #550: update download by tag.
- Fix Issue #548: encode filename to utf-8 if required.
- Fix Issue #540: skip password protected zip file checking.
- Update #505: update error message for reCAPTCHA.
- Implement #544: support for FANBOX post type = entry.
- Fix Issue #556: update parser for image and member.
- Fix Issue #558: update write json function.
- Fix Issue #559: fix tags encoding for option 10.
- Fix Issue #555: skip import to db if list.txt doesn't exists.
- Merge #563: perform FS with utf8 support check only once.
- Fix Issue #562: Fix encoding when writing info file.
20190919
- Fix Issue #530: update OAuth login header.
20190907b
- Implement #518: show asterisk '*' when entering password on console for Windows.
- Implement #521: skip tags if exceeding limit defined in 'checkUpdatedLimit'. 'alwaysCheckFileSize' need to be disabled.
- Merge #513 and #519
- Fix Issue #522: rearrange import for getpass.
20190712
- Implement #490: Show current and total member id in the title.
- Fix Issue #503: skip remote file size if server return error 404 or 500.
- Fix Issue #508: update logic for background image download.
20190617-beta1
- Fix Issue #500: add 'google_forms' for embed provider and update max page parser.
- Update the embedData parser to use configuration in content_provider.json.
20190606
- Fix Issue #495: Update FANBOX parser for supported artist.
20190529
- Fix Issue #494: update fanbox next page url handling.
- Add option to skip SSL verification for OAuth (useful in debugging).
20190523
- Update #493: update fanbox parser to support vimeo content provider.
20190523
- Implement #493: update fanbox parser to support fanbox content provider.
20190501
- Fix Issue #483: handle oAuth to get the member information for FANBOX, use WEB-RPC to get member info if image id available.
- Fix Issue #481: update write image info for json.
- Fix Issue #485: update parser for sorting parameter.
- Update readme.txt
- Update requirements for running from source code.
20190416
- Implement #468: add sorting order for download bookmarked images.
- Implement #480: update API url.
20190401
- Implement #476: FANBOX support for video post.
20190321
- Update #447: change filenameMangaFormat to filenameformat.
20190319
- Implement #475: support youtube for embedMap.
- Update #447: allow to write info text even if no images.
20190316
- Implement #447: Write FANBOX post info to text file.
20190301
- Implement #470: add support for embedMap for twitter.
- Fix Issue #472: Update search by tags parser for null illustId.
20190212
- Fix Issue #466: Update handler for deleted member.
- Fix Issue #467: Add retry for pixiv FANBOX handler.
20190120
- Fix Issue #461: ugoira to gif.
20190119
- Fix Issue #457: update ugoira handler.
20190108
- Implement #455: Add Animated WebP support for ugoira.
- Implement #451: Update file already downloaded check for ugoira.
- Move ugoira related configuration to separate section in config.ini
- Change the default parameter for WebM creation to '-lossless 1 -vsync 2 -r 999' to support variable fps.
20181212
- Implement #445: update parser for FANBOX post.
- Implement #449: enable time format for filename format=%date_fmt{format}%
- Implement #451: update ugoira creation handler if zip file already exists.
20181121
- Implement #438: update Fanbox parser for article post.
- Update test script for pytest.
- Update readme.txt.
20181105
- Fix Issue #432: Fix parser for suspended users.
- Fix Issue #429: Fix parser for user background image.
- Implement #423: Update command line switches for FANBOX.
- Add new config to check latest version in github.
- set `checknewversion = False` in the `[Network]` section at config.ini
- Update readme.txt
20181008
- Implement #402: allow download from pixiv Fanbox.
- Use 'f1' to download from the artists you following.
- Use 'f2' to download specific artist you want to download. Please note you need to follow the artist Fanbox, else it will only get the free post.
- Limitation:
- Somehow the file size detection doesn't works, so the overwrite feature is disabled.
- The original url is modified for compatibility when generating the local filename.
- Currently only support 'text', 'image', and 'file' post type.
- Implement #420: enable timezone adjustment for works date. Set uselocaltimezone = True to enable in config.ini
- Fix Issue #421: update parser for url in description.
20180928
- Fix Issue #415: change 'or' to 'and', reformat code to conform pep8.
- Fix Issue #416: sort the imagelist as int.
- Merge #410, #411, and #413 from @awiebe.
20180918
- Implement #394: update parser for artist info and images using ajax api.
- Fix Issue #399: update the total images display.
- Fix Issue #400: update parser for artist if artist doesn't exists.
- Fix Issue #404: update parser for artist bookmarks.
- Fix Issue #409: update parser for download by artist id and tag.
- Add API call caching with 1 hour expiry.
- Update test cases.
20180726
- Fix Issue #388: Update parser for artist info.
- Implement #373: artist background image will be downloaded together with artist avatar.
- Update test cases.
20180702
- Fix Issue #369: Update parser for member token if using option 2.
- Fix Issue #374: Update datediff handler to include utc timezone.
- Fix Issue #381: Fix Ugoira2Webm converter.
20180607
- Fix Issue #355: Initial support for new ui.
- Fix Issue #257: update filename scope in process_image.
- Fix Issue #360: enable option 11 on new ui.
- Fix Issue #364: switch to demjson to parse js object for new ui.
- Fix Issue #372: update ugoira parser.
- Code update based on PyLint suggestion.
20180414
- Fix Issue #337: failed to update cookie value.
- Implement Feature #349: enhance command line option for -s e and -s m.
20180316
- Fix Issue #343: Update image parser.
20180314 WhiteDay
- Implement #40: Enhance error message to include current page number for tags and member.
- Fix Issue #194: Add retry when exporting bookmark.
- Fix Issue #294: Update root path for both image and manga pages.
- Implement #314: Set non-zero exit code for invalid input.
- Fix Issue #315: Update parser for member token.
- New Setting: setLastModified = True
Set last-modified and last-accessed timestamp based on work's timestamp.
- Implement #257: convert ugoira to webm.
Require ffmpeg and createUgoira = True
- New Settings under [FFmpeg] section:
- ffmpeg = "ffmpeg"
FFmpeg path
- ffmpegCodec = "libvpx-vp9"
Codec to use
- ffmpegParam = "-lossless 1"
Additional Parameters
- Fix Issue #332
- Merge #328: Feature/python3 print format
- Fix #339: Local file checking logic.
- Fix #340: Database file cleanup check.
- Fix #341: Update image parser.
- Update test cases.
20180218
- Implement #243: Add configuration to delete ugoira after converting to gif/apng. (Set deleteUgoira to True in config.ini).
20180118
- Fix Issue #305: Temporarily fixed until new win_unicode_console is updated from 0.5.
- Implement #303: Increase SQLITE timeout to 5minutes.
- Implement #299: Enhance alwaysCheckFileSize logic to check local file exists first before getting remote filesize.
- Fix #300: update parser for downloading own images
- Implement #296: Code reformating to PEP8
- Update progress report.
20171226:
- Fix Issue #290: Update parser for option 8. New image from followed members.
20171127:
- Implement #278: Allow to pass tag from command line operation for option 6.
- Implement #276: Add random delay between each image post, set downloadDelay=0 to disable in config.ini.
- Fix Issue #277: Update file sanitazion logic. Always replace '/' and '\' for %title%, %artist%, %searchTags%, %tags%, %works_tools%, and %original_artist% to '_'.
20171006:
- Fix Issue #272: fix PIL import
20171004:
- Update test cases.
- Fix Issue #252: update parser for tags search.
- Fix Issue #261: fix r18 url.
- Fix Issue #262: support downloading your own works (artist info will not be downloaded).
- Fix Issue #270: update parser for tags search again.
- Implement #250: support to specify timeframe for download by taglist.
- updated readme.txt
20170705:
- Update test cases.
- Fix Issue #222: hide password in console.
- Fix Issue #223: fix db result error.
- Fix Issue #224 and #226: new big image mode for single page manga.
- Fix Issue #228: Update urls to https.
- Fix Issue #230: set defaultencoding to utf-8.
- Fix Issue #232: UnboundLocalError local variable 'list_page' referenced before assignment.
- Fix Issue #233: update dump hmtml page handler.
- Fix Issue #236: update parser for artist profile and dump html page if not found.
- Fix Issue #237: update UnboundLocalError for list_page.
- Fix Issue #238: update DB clear method for ugoira/zip file.
- Update readme.txt
20170411:
- Update test cases.
- Fix Issue #218: handle mechanize lib update.
- Fix Issue #219: update parser for option 11 (download member bookmarks).
- Fix Issue #220: update parser for option 6 (download user bookmarked images).
20170220:
- Update test cases.
- Fix Issue #178: Unbound variable upon error handling.
- Fix Issue #192: Syntax error :P.
- Fix Issue #195: Update imageio library url.
- Fix Issue #196: Allow any whitespace after memberid in list.txt
- Fix Issue #200: Update tag parsing.
20170131:
- Fix code to enable dumping medium page by configuration.
- Try to Fix Issue #182: Handle missing stats on Japanese UI.
- Implement #184: Always send charset header value as utf-8.
- Fix Issue #191: handle custom directory in list.txt
20161231
- Fix Issue #181: Login using password.
20161207
- Fix download by list.txt and tags for oldui.
20161205
- Implement #149: APNG generation. Set createApng = True in config.ini to enable.
- Implement #172: Modify option 4 to allow download by member id list and tag.
- Implement #174: Allow custom avatar name format, refer to readme.txt for available format. Set blank to use default behaviour.
- implement #177: Allow custom current date format, refer to readme.txt for formating syntax.
20161124
- Fix Issue #173: Option 11 for oldui and whitecube.
- Update ugoira support for whitecube.
- Update option 10 support for whitecube.
20161113b
- Fix Issue #168: Fix referer for avatar image download.
- Implement whitecube search tag mode.
- Fix Issue #170: fix PixivHelper import.
- Add call to use win_unicode_console, you need to set unicode-supported font in the windows console properties.
- fix folder.jpg filename generator.
20161012
- Fix Issue #152: fix UnboundLocalError for image_id.
- Implement #145: enable ideographic space (\u3000) in tag separator, use %ideo_space% in config.ini.
20160927
- Fix Issue #147: update parser to handle invalid member id.
20160814
- Fix Issue #143: update parser to handle specific tags.
20160806
- Implement #141: allow custom datetime format for filename.
- Fix image caption parsing.
- Remove tempFolder configuration, use tempfile.mkdtemp() instead.
20160726
- Implement #135: convert ugoira to gif.
- Implement #140: skip image based on blacklisted member id.
20160711
- Implement #55: update list.txt parser to support member id extraction from url.
20160613
- Fix Issue #116: update login method.
20160513
- Add null check before dumping html.
- Fix Issue #115: update login method.
20160421
- Implement #108: enhance window titlebar to calculate the total images.
- Fix Issue #112: change the max page limit for option 8 from 100 to 5000 for premium user.
- Fix Issue #113: update filename sanitizer for windows.
- Fix Issue #114: change login url to https://www.pixiv.net/login.php.
20160319
- Fix unicode decode error when downloading image.
- Implement #107: add option to specify different database.
- Update readme.txt
- Fix Issue #109: update parser for title and caption.
- Fix Issue #108: parse total image count.
20160211:
- Fix Issue #98: Fix error message when download fail.
- Add option to filter artist image bookmark by tags.
- Update parser.
- Fix Issue #103: Fix overwrite local file option.
- Fix Issue #104: Fix %artist% parser.
- Implement Feature #105: Download bookmarked image by tag.
20151112:
- Fix Issue #96: page 100 is not downloaded for new illust page.
- Update test page.
- Implement Feature #95: dump url list to text file.
Set writeUrlInDescription = True to enable.
Set urlBlacklistRegex to define url to ignore.
Set urlDumpFilename to define the dump filename, use python strftime() format.
20151019:
- Update proxy handler.
- Add download file verifier (image and ugoira).
- Optimize filesize check.
- Update big manga parser.
- Fix Issue #94: use correct url for R-18 new illust.
20151001:
- Fix Issue #93: remove popular introduction from tags search result.
20150924:
- Fix Issue #88: undefined variable when handling cookie.
- Externalize date format to config.ini, refer to http://strftime.org/ for formatting syntax.
20150715:
- Fix Issue #87: Update login logic, now always use https.
20150711:
- Implement #85: Enable infinite loop for download by tags.
- Fix Issue #79: Don't save if the download is not complete.
- Add option to dump tags search page.
- Reorganize config.ini.
20150605:
- Merge pull request #76 from woky/master: Expand ~ in path options.
- Add option to filter images by bookmark count for download by taglist (console input only).
- Fix Issue #81: Divided by Zero when download time is instant.
20150427:
- Add feature to download both private and public image bookmark.
- Fix Issue #70: Update Error code updated and use sys.exit() instead of os._exit()
- Fix Issue #74: add check based on saved filename, old manga data might need to be redownloaded.
- Add ability to export list of user bookmarks for user with given member_id.
- Pull Request #73: Merge export user bookmarks from prototype27/topic/exportUserBookmarksForUser.
20150328:
- Implement #69: Create .ugoira format, open with http://www.bandisoft.com/forum/viewtopic.php?f=8&t=3359
- Implement #69: Add local .ugoira file checking.
- Implement #70: call os._exit() with error code != 0 if got exception.
- Fix Issue #71: Always overwrite files even if local file is larger, use backupoldfile=True to backup the old file!
- Fix Issue #72: Update download by tags parser.
- Fix Issue #69: Add option to delete zip file, set deleteZipFile = True.
20150218:
- Fix Issue #64: Update login form selector.
20150121:
- Implement #66: Add option to download from oldest image first for Download by Tags/Tags list.
20150105:
- Update the SSL login page url to https://www.secure.pixiv.net/login.php.
- Update BeautifulSoup to version 3.2.1.
20141216:
- Fix Issue #63: Update image parser again :(.
20141212b:
- Fix Issue #60: Update image parser to handle old and new layout.
- Fix html dump logic.
20141201:
- Fix Issue #58: Update handler for private image.
- Update handler for deleted image.
20141121:
- Fix issue #57: Type error when downloading from tags.
20141006:
- Ignore userobots setting, now is set to always False.
- Update manga parser again.
20141004c:
- Update parser.
- Update download logic for manga page (20141004b)
- Update parser for the new 2 page manga mode (20141004c)
20140926
- Fix Issue #47: Update manga parser.
20140916
- Fix Issue #44: process image to handle known image exception.
- Implement #43: write ugoira data to separate file.
20140712
- Fix parser for Option 8: Download new illust from bookmark.
20140705
- Update download by member id to accept comma-separated list from console.
- Update download by image id to accept comma-separated list from console.
20140628
- Update parser to support animation (ugoira_view).
20140621
- Update artist token parser.
20140620
- Fix issue #36: Member illustration parser update.
- Fix issue #37: Bookmark parser update.
20140606
- Fix issue #33: fallback to member token if nickname not found
- Add CLI option to load config file from path.
- Implement feature #30: use %bookmark_count% and %image_response_count%
20140325
- Update unknown error handler.
- Add %bookmark_count% for filename format, only available for download by tags.
20140222
- Update model.
- Update test cases.
20140126
- Fix download by Tags.
20131218
- Add 403 handler when processing member.
- Fix Japanese worksdate parser.
20131214
- Add 404 handler when processing member.
- Reset selection when Ctrl-C pressed.
- Reindent and encode source code to UTF-8.
20131122
- Fix Unicode error for Download by Tag.
- Add feature to replace the root path of image in database.
20131005
- Fix image parsing.
- Update test case.
- Add new option dumpMediumPage for debugging.
20130927
- Change some command line syntax:
- Download by Tags (3): wilcard[y/n], start page, end page, tags
- Download by Title Caption (9): start page, end page, title/captions
- Download by Tags and MemberId (10): MemberId, start page, end page, tags
- Download by Group ID (12): GroupId, limit, process external[y/n]
- Fix trailing space before extensions in filename
- Add SOCKS proxy support, use socks5:// or socks4:// format.
- Add temporary fix for datetime parsing (pixiv server bug).
20130912
- Add option to disable html dump and skipDumpFilter.
- Update console title for download by member and download by tags.
- Add download by group id.
20130804
- Move updatedLimitCount to outer loop of member.
20130714
- Add logLevel option.
- Fix UnboundLocalError for time as reported by Janice.
- Add handler for server error: "An error has occured".
20130705
- Update manga parser.
- Update caption parser.
- Add new option to back up old file if the filesize is different (backupOldFile).
- Auto Backup old config.ini if error detected (e.g.: missing entries) to config.ini.error-xxx.
- Update test cases.
20130511
- Refactor out PixivException and PixivBrowserFactory.
- Update error codes.
- Update artist token parser to handle all img-inf in image list.
- Update filename sanitizer to remove all Unicode Control Character (Cc).
20130430
- Update parser.
- Add start/end page for Download from Bookmarked Members (option 5).
- Remove member id from option 6, use option 11, Download bookmarked images by member_id.
- Add url logging for failed login using cookie.
20130330
- Add Ctrl+C handler for download by member id and download by tags to
allow user to skip to next image or abort the whole process.
20130327
- Add option to keep signed in, set keepsignedin = 1 in config.ini on
[Authentication] section.
- Add login error message parser.
20130322
- Add download time and speed.
- Backup config.ini when saving to config.ini.bak.
20130321
- Fix command line option for download by member id and tags to allow single
tag search.
20130316
- Add option to skip image if worksDate < dateDiff, this will skip to next
member, or stop processing for download from new bookmark.
20130308
- Add menu to reload config.ini
- Add %date% for filename format (YYYYMMMDD).
20130222
- Update parser to the latest pixiv layout.
- Update last page detection (20130222b).
- Update sanitizeFilename: remove new line marker '\r\n' (20130222c).
20130201
- Update imageDownload to skip if got 502 error.
- Fix end page detection for download by tags.
20130128
- Fix end page limit.
- Update parser for pixivArtist and download by tags.
20121215
- Fix title, tags, and caption parsing.
- Add new filename format for download by tags:
-%searchTags% ==> for download by tags, add searched tag keyword.
20121208
- Update no image detection for member
- Fix issue #14: ignore showcase for download by tag
- Fix issue #15: Update query for search by tag
- Add R-18 mode in search by tag
20121124
- Update error detection for image page.
- Add R18 mode only for member/member's bookmarks download.
- Set r18mode = True to apply.
- Fix page limit detection.
20121108
- Add feature for download member's bookmarked images.
- Add new filename format for member's bookmark mode.
- %bookmark% ==> for bookmark mode, add 'Bookmarks' string.
- %original_member_id% ==> for bookmark mode, put original member id.
- %original_member_token% ==> for bookmark mode, put original member token.
- %original_artist% ==> for bookmark mode, put original artist name.
20121102
- Add feature to export image information to text file.
- Set writeimageinfo = True to use.
20121028
- Update error message detection for MyPick and Ero/Guro page.
20121013
- Fix irfanView slide show.
20121006
- Fix unbound variable error.
- Update parser to check if user logged in.
20120929
- Update medium image parser to handle no tags.
20120928
- Update medium image page parser for tags and unit test.
20120923
- Fix PixivConfig doesn't allow %page_number% in settings.
20120921
- Fix login method to use correct form.
20120903
- Add option to compact database.
- Format %works_date*% to use dash '-', instead of slash '/'.
20120831
- Update error detection for new error messages.