diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..8f71f43 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/README.MD b/README.MD index 319cd42..dd72a93 100644 --- a/README.MD +++ b/README.MD @@ -1,8 +1,25 @@ -[![Build Status](https://travis-ci.org/c-martinez/ShiCo.svg?branch=master)](https://travis-ci.org/c-martinez/ShiCo) +[![Build Status](https://travis-ci.org/NLeSC/ShiCo.svg?branch=master)](https://travis-ci.org/NLeSC/ShiCo) -# ShiCo package +# ShiCo -- Exploring Shifting Concepts Through Time -More documentation will follow +## What is ShiCo? + +ShiCo is a tool for visualizing time shifting concepts. We refer to a concept as the set of words which are related to a given seed word. ShiCo uses a set of semantic models (word2vec) spanning a number of years to explore how concepts change over time -- words related to a given concept at time *t=0* may differ from the words related to the same concept at time *t=n*. For example: + +![Mock concept shift](./docs/mockConcept1.png) +![Mock concept shift](./docs/mockConcept2.png) + +You can find more details of how the concept shift works [here](./docs/howItWorks.md). + + - *How is it structured (backend/frontend)* + - *What the back end does?* + - *What do I see in the front end?* + +*How to use it?* + - *What do I need to run it? (python, some web server, word2vec models)* + +*How to extend it?* + - *Use different semantic model (other than word2vec)* ## Launching server @@ -62,3 +79,7 @@ $ nosetests ## Speeding up ShiCo Current implementation of ShiCo relies on gensim word2vec model `most_similar` function, which in turn requires the calculation of the dot product between two large matrices, via `numpy.dot` function. For this reason, ShiCo greatly benefits from using libraries which accelerate matrix multiplications, such as OpenBLAS. ShiCo has been tested using [Numpy with OpenBLAS](https://hunseblog.wordpress.com/2014/09/15/installing-numpy-and-openblas/), producing a significant increase in speed. + +## Licensing + +Source code and data of ShiCo is licensed under the Apache License, version 2.0. diff --git a/docs/howItWorks.md b/docs/howItWorks.md new file mode 100644 index 0000000..ca93ea5 --- /dev/null +++ b/docs/howItWorks.md @@ -0,0 +1,91 @@ +# How does ShiCo work? + +Given a set of seed terms, ShiCo uses its semantic models to generate a vocabulary of related terms. This process is done for every one of the semantic models available. This is done using a *Vocabulary Monitor*. The vocabulary monitor generates a vocabulary of related terms and a list of vocabulary links, which explains which of the related terms was generated by which one of the seed terms. + +![Overview of vocabulary monitor](./shicoOverviewA.png) + +The produced vocabularies and links are then aggregated using a *Vocabulary Aggregator* -- the vocabulary aggregator groups together results from multiple models into a single final vocabulary. This has a smoothing effect on the produced vocabulary. + +![Overview of vocabulary aggregator](./shicoOverviewB.png) + +## Vocabulary Monitor + +The vocabulary monitor uses the given terms to query the semantic models. The models provide a list of *related words* and some measure of how closely related they are to the seed term -- this is interpreted as a *distance* between the seed term and the related word: + +![Overview of vocabulary aggregator](./seedWordRelationA.png) + +As mentioned before, from each model the vocabulary monitor generates a vocabulary and list of vocabulary links. The vocabulary is a list of terms related to the seed terms -- each term is assigned a *weight*. This weight is calculated in one of two ways: + - As a count of the number of times the term appears + - As a sum of the distance between the seed terms and the related term + +These weightings start from the assumption that a related term can appear as the result of one or more seed terms, in which case each time it increases the weight assigned to the related term. + +![Overview of vocabulary aggregator](./seedWordRelationB.png) + +In this way, the weights of each word are the sum of the weight contribution from each seed. + +``` +weights(word1) = weight(word1,seed1) +weights(word2) = weight(word2,seed1) + weight(word2,seed2) +... +``` + +The vocabulary monitor limits the number of words in the generated vocabulary, only the *N* words with the highest weights are included in the vocabulary. + +``` +word1: weights(word1) +word2: weights(word2) +... +wordN: weights(wordN) +-------- cut point +wordN+1: weights(wordN+1) # dropped +wordN+2: weights(wordN+2) # dropped +... # dropped +``` + +### Algorithms + +The vocabulary monitor contains can use two different algorithms for generating vocabularies. These control which words are used as seed terms for each model. + +#### Non-adaptive + +The non-adaptive vocabulary generator uses the same seed terms each time to generate the related terms. + +![Non Adaptive generator](./vmNonAdaptive.png) + +#### Adaptive + +The adaptive vocabulary generator uses the related terms generated by one semantic model as seed terms for the next semantic model. + +![Adaptive generator](./vmAdaptive.png) + +This adds an additional possibility: it allows for the semantic models to be used in chronological order, or in reverse chronological order -- searching forwards or backwards in time. + +For more details, please refer to [this paper](#Wevers DH2015). + + +## Vocabulary Aggregator + +A vocabulary aggregator takes a vocabulary produced by a vocabulary monitor and aggregates them over a set time window. + +![Vocabulary aggregator](./vaYears.png) + +Weighting functions are used to aggregate topics in a year: + +Terms inside the 'window', are weighted by a weighting function. + +### Gaussian +![Gaussian weighting function](./vaWeightGauss.png) + + +### JSD +![JSD weighting function](./vaWeightJSD.png) + +### Linear +![Linear weighting function](./vaWeightLinear.png) + +For more details, please refer to [this paper](#Wevers DH2015). + +# References +##### Wevers DH2015 +Wevers, M.; Kenter, T. and Huijnen, P. Concepts Through Time: Tracing Concepts in Dutch Newspaper Discourse (1890-1990) using Word Embeddings. In Digital Humanities 2015 (DH2015), 2015. diff --git a/docs/mockConcept1.png b/docs/mockConcept1.png new file mode 100644 index 0000000..ad39c66 Binary files /dev/null and b/docs/mockConcept1.png differ diff --git a/docs/mockConcept2.png b/docs/mockConcept2.png new file mode 100644 index 0000000..11fa3f9 Binary files /dev/null and b/docs/mockConcept2.png differ diff --git a/docs/seedWordRelationA.png b/docs/seedWordRelationA.png new file mode 100644 index 0000000..13f247a Binary files /dev/null and b/docs/seedWordRelationA.png differ diff --git a/docs/seedWordRelationB.png b/docs/seedWordRelationB.png new file mode 100644 index 0000000..6300128 Binary files /dev/null and b/docs/seedWordRelationB.png differ diff --git a/docs/shicoOverviewA.png b/docs/shicoOverviewA.png new file mode 100644 index 0000000..4376ab0 Binary files /dev/null and b/docs/shicoOverviewA.png differ diff --git a/docs/shicoOverviewB.png b/docs/shicoOverviewB.png new file mode 100644 index 0000000..2db8aac Binary files /dev/null and b/docs/shicoOverviewB.png differ diff --git a/docs/vaWeightGauss.png b/docs/vaWeightGauss.png new file mode 100644 index 0000000..6f28ce1 Binary files /dev/null and b/docs/vaWeightGauss.png differ diff --git a/docs/vaWeightJSD.png b/docs/vaWeightJSD.png new file mode 100644 index 0000000..0805fc7 Binary files /dev/null and b/docs/vaWeightJSD.png differ diff --git a/docs/vaWeightLinear.png b/docs/vaWeightLinear.png new file mode 100644 index 0000000..001ec4d Binary files /dev/null and b/docs/vaWeightLinear.png differ diff --git a/docs/vaYears.png b/docs/vaYears.png new file mode 100644 index 0000000..a9e6fba Binary files /dev/null and b/docs/vaYears.png differ diff --git a/docs/vmAdaptive.png b/docs/vmAdaptive.png new file mode 100644 index 0000000..a67b53f Binary files /dev/null and b/docs/vmAdaptive.png differ diff --git a/docs/vmNonAdaptive.png b/docs/vmNonAdaptive.png new file mode 100644 index 0000000..a26eaa1 Binary files /dev/null and b/docs/vmNonAdaptive.png differ diff --git a/shico/format.py b/shico/format.py index 25002ce..71505cc 100644 --- a/shico/format.py +++ b/shico/format.py @@ -35,14 +35,14 @@ def yearTuplesAsDict(results): E.g. From: { - 1950: ('a',1), ('b',2),('c',3) + 1950: ('a',w1), ('b',w2),('c',w3) } To: { 1950: { - 'a': 1, - 'b': 2, - 'c': 3 + 'a': w1, + 'b': w2, + 'c': w3 } } ''' @@ -64,29 +64,29 @@ def _buildNode(word, counts, seedSet, finalWords): } -def _buildLink(seed, word, weight, nodeIdx): +def _buildLink(seed, word, distance, nodeIdx): ''' Build a node for a force directed graph in format used by front end''' seedIdx = nodeIdx[seed] wordIdx = nodeIdx[word] return { 'source': seedIdx, 'target': wordIdx, - 'value': 1 / (weight + 1) + 'value': 1 / (distance + 1) } def _buildLinks(yLinks, nodeIdx): '''Build a list of links from the given yearly links. For each year group, link all seeds to all their results (with strength proportional to their - weight)''' + distance)''' linkList = [] # We are not doing anything with the years in yLinks for links in yLinks.values(): for seed, results in links.iteritems(): - for word, weight in results: + for word, distance in results: # TODO: check seeds present in dict more elegantly if seed in nodeIdx and word in nodeIdx: - linkList.append(_buildLink(seed, word, weight, nodeIdx)) + linkList.append(_buildLink(seed, word, distance, nodeIdx)) else: print 'Seed or word not in index!' return linkList diff --git a/shico/server.py b/shico/server.py index b0d27cf..aa39d6a 100644 --- a/shico/server.py +++ b/shico/server.py @@ -11,7 +11,7 @@ from docopt import docopt from flask import Flask, jsonify -from flask_restful import reqparse, inputs +from flask_restful import reqparse from flask.ext.cors import CORS from vocabularymonitor import VocabularyMonitor @@ -33,6 +33,29 @@ def validatestr(value): except: raise ValueError +def isValidOption(value, options): + '''Validate that given value is a string from a predetermined set. + Used to validate tracker parameters.''' + if value in options: + return value + else: + raise ValueError + +def validAlgorithm(value): + '''Validate algorithm -- in lower case''' + return isValidOption(value, _algorithms).lower() + +def validWeighting(value): + '''Validate weighting function''' + return isValidOption(value, _weighFuncs) + +def validDirection(value): + '''Validate direction is Forward (false means backward)''' + return isValidOption(value, _directions)=='Forward' + +def sumSimilarity(value): + '''Validate boost methods is Sum distances (false means Counts)''' + return isValidOption(value, _boostMethods)=='Sum similarity' def initApp(files, binary): '''Initialize Flask app by loading VocabularyMonitor. @@ -54,16 +77,21 @@ def initApp(files, binary): trackParser.add_argument('endKey', type=validatestr, default=None) trackParser.add_argument('minDist', type=float, default=0.0) trackParser.add_argument('wordBoost', type=float, default=1.0) -trackParser.add_argument('forwards', type=inputs.boolean, default=True) -trackParser.add_argument('sumDistances', type=inputs.boolean, default=False) -trackParser.add_argument('algorithm', type=str, default='adaptive') +trackParser.add_argument('forwards', type=validDirection, default=True) +trackParser.add_argument('boostMethod', type=sumSimilarity, default=True) +trackParser.add_argument('algorithm', type=validAlgorithm, default='adaptive') # VocabularyAggregator parameters: -trackParser.add_argument('aggWeighFunction', type=str, default='Gaussian') +trackParser.add_argument('aggWeighFunction', type=validWeighting, default='Gaussian') trackParser.add_argument('aggWFParam', type=float, default=1.0) trackParser.add_argument('aggYearsInInterval', type=int, default=5) trackParser.add_argument('aggWordsPerYear', type=int, default=10) +_algorithms = ('Adaptive', 'Non-adaptive') +_weighFuncs = ('Gaussian', 'Linear', 'JSD') +_directions = ('Forward', 'Backward') +_boostMethods = ('Sum similarity', 'Counts') + @app.route('/available-years') def avlYears(): @@ -92,7 +120,7 @@ def trackWord(terms): minDist=params['minDist'], wordBoost=params['wordBoost'], forwards=params['forwards'], - sumDistances=params['sumDistances'], + sumSimilarity=params['boostMethod'], algorithm=params['algorithm'], ) agg = VocabularyAggregator(weighF=params['aggWeighFunction'], @@ -102,7 +130,6 @@ def trackWord(terms): ) aggResults, aggMetadata = agg.aggregate(results) - # TODO: use used seeds for next loop query networks = yearlyNetwork(aggMetadata, aggResults, results, links) return jsonify(stream=yearTuplesAsDict(aggResults), networks=networks) diff --git a/shico/vocabularyaggregator.py b/shico/vocabularyaggregator.py index c5bfaa2..895eda4 100644 --- a/shico/vocabularyaggregator.py +++ b/shico/vocabularyaggregator.py @@ -68,7 +68,9 @@ def aggregate(self, vocab): def _adaptiveAggregation(V, n, yIntervals, weightF, param, freq): - '''Apply adaptive aggregation algorithm to the given vocabulary.''' + '''Apply adaptive aggregation algorithm to the given vocabulary. + Algorithm 2 from paper. + ''' # Initialize returned parameters finalVocabs = SortedDict() periodGroups = SortedDict() diff --git a/shico/vocabularymonitor.py b/shico/vocabularymonitor.py index 4415483..51563ff 100644 --- a/shico/vocabularymonitor.py +++ b/shico/vocabularymonitor.py @@ -55,7 +55,7 @@ def getAvailableYears(self): def trackClouds(self, seedTerms, maxTerms=10, maxRelatedTerms=10, startKey=None, endKey=None, minDist=0.0, wordBoost=1.00, - forwards=True, sumDistances=False, algorithm='adaptive'): + forwards=True, sumSimilarity=False, algorithm='adaptive'): '''Given a list of seed terms, generate a set of results from the word2vec models currently loaded in this vocabularymonitor. @@ -73,7 +73,7 @@ def trackClouds(self, seedTerms, maxTerms=10, maxRelatedTerms=10, wordBoost -- Weight boost automatically given to seed terms. forwards -- Perform search in the forward time direction. Set to False for backward direction. - sumDistances -- Use the 1-distance as weighting factor. If set to + sumSimilarity -- Use the 1-distance as weighting factor. If set to False, weighting is 1 for each occurrence. algorithm -- 'adaptive' or 'non-adaptive' algorithm (adaptive previously known as inlinks). Adaptive algorithm @@ -120,7 +120,6 @@ def trackClouds(self, seedTerms, maxTerms=10, maxRelatedTerms=10, raise KeyError('Key ' + startKey + ' not a valid model index') keyIdx = sortedKeys.index(startKey) sortedKeys = sortedKeys[keyIdx:] - # Select end key if (endKey is not None): if endKey not in sortedKeys: @@ -141,7 +140,7 @@ def trackClouds(self, seedTerms, maxTerms=10, maxRelatedTerms=10, maxRelatedTerms=maxRelatedTerms, minDist=minDist, wordBoost=wordBoost, - sumDistances=sumDistances) + sumSimilarity=sumSimilarity) elif algorithm == 'non-adaptive': # Non-adaptive algorithm uses always same set of seeds terms, links = \ @@ -159,9 +158,9 @@ def trackClouds(self, seedTerms, maxTerms=10, maxRelatedTerms=10, return yTerms, yLinks def _trackInlink(self, model, seedTerms, maxTerms=10, maxRelatedTerms=10, - minDist=0.0, wordBoost=1.0, sumDistances=False): + minDist=0.0, wordBoost=1.0, sumSimilarity=False): '''Perform in link search''' - if sumDistances: + if sumSimilarity: terms, links = self._trackCore( model, seedTerms, maxTerms=maxTerms, maxRelatedTerms=maxRelatedTerms, minDist=minDist, diff --git a/webapp/bower.json b/webapp/bower.json index 1fe0913..9d571bf 100644 --- a/webapp/bower.json +++ b/webapp/bower.json @@ -14,7 +14,8 @@ "angular-animate": "~1.4.8", "angular-busy": "~4.1.3", "nvd3": "https://github.com/c-martinez/nvd3.git#myrelease", - "ng-csv": "^0.3.6" + "ng-csv": "^0.3.6", + "angular-marked": "^1.2.0" }, "devDependencies": { "angular-mocks": "~1.4.2" diff --git a/webapp/gulp/build.js b/webapp/gulp/build.js index 6476e03..bb6ff66 100644 --- a/webapp/gulp/build.js +++ b/webapp/gulp/build.js @@ -55,7 +55,7 @@ gulp.task('html', ['inject', 'partials'], function () { return gulp.src(path.join(conf.paths.tmp, '/serve/*.html')) .pipe($.inject(partialsInjectFile, partialsInjectOptions)) .pipe(assets = $.useref.assets()) - .pipe($.rev()) + //.pipe($.rev()) .pipe(jsFilter) .pipe($.sourcemaps.init()) .pipe($.ngAnnotate()) @@ -69,7 +69,7 @@ gulp.task('html', ['inject', 'partials'], function () { .pipe(cssFilter.restore) .pipe(assets.restore()) .pipe($.useref()) - .pipe($.revReplace()) + //.pipe($.revReplace()) .pipe(htmlFilter) .pipe($.minifyHtml({ empty: true, diff --git a/webapp/package.json b/webapp/package.json index e7540cd..123a3b7 100644 --- a/webapp/package.json +++ b/webapp/package.json @@ -25,8 +25,6 @@ "gulp-ng-annotate": "~1.1.0", "gulp-replace": "~0.5.4", "gulp-rename": "~1.2.2", - "gulp-rev": "~6.0.1", - "gulp-rev-replace": "~0.4.2", "gulp-minify-html": "~1.0.4", "gulp-inject": "~3.0.0", "gulp-protractor": "~1.0.0", diff --git a/webapp/src/app/components/shico/conceptTracker.service.js b/webapp/src/app/components/shico/conceptTracker.service.js index fa4c007..c7fcc24 100644 --- a/webapp/src/app/components/shico/conceptTracker.service.js +++ b/webapp/src/app/components/shico/conceptTracker.service.js @@ -21,6 +21,13 @@ function parseTermTrack(data) { // If data needs to be parsed, it should be done here. + + // Copy year into each node + angular.forEach(data.networks, function(net,year) { + angular.forEach(net.nodes, function(node) { + node.year = year; + }); + }); return data.toJSON(); } } diff --git a/webapp/src/app/components/shico/graphConfig.service.js b/webapp/src/app/components/shico/graphConfig.service.js index 655c6e8..b0bc1c5 100644 --- a/webapp/src/app/components/shico/graphConfig.service.js +++ b/webapp/src/app/components/shico/graphConfig.service.js @@ -93,7 +93,7 @@ // Helper functions for forceConfig function processNode(node) { - // Nodes have: {'name': 'str', 'type': 'seed', 'count': N}, + // Nodes have: {'name': 'str', 'type': 'seed', 'count': N, year: XXXX}, addTextLabels(node); setSize(node); angular.forEach(forceGraphHooks, function(hook) { diff --git a/webapp/src/app/components/shico/parameterIO.directive.js b/webapp/src/app/components/shico/parameterIO.directive.js index 26fcdd3..9dbec96 100644 --- a/webapp/src/app/components/shico/parameterIO.directive.js +++ b/webapp/src/app/components/shico/parameterIO.directive.js @@ -8,7 +8,7 @@ function parameterIO() { var directive = { scope: {}, // Directive has it's own personal scope - templateUrl: '/app/components/shico/parameterIO.template.html', + templateUrl: 'app/components/shico/parameterIO.template.html', controllerAs: 'vm', controller: 'ParameterIOController' }; diff --git a/webapp/src/app/components/shico/trackerGraphs.controller.js b/webapp/src/app/components/shico/trackerGraphs.controller.js index 59606d4..d8009f0 100644 --- a/webapp/src/app/components/shico/trackerGraphs.controller.js +++ b/webapp/src/app/components/shico/trackerGraphs.controller.js @@ -19,8 +19,8 @@ vm.downloadData = downloadData; function yearsInSight(yearIdx) { - return (vm.forceGraph.currYearIdx - 2) <= yearIdx && - yearIdx < (vm.forceGraph.currYearIdx + 2); + return (vm.forceGraph.currYearIdx - 1) <= yearIdx && + yearIdx <= (vm.forceGraph.currYearIdx + 1); } function addBorder(scope) { diff --git a/webapp/src/app/components/shico/trackerGraphs.directive.js b/webapp/src/app/components/shico/trackerGraphs.directive.js index 0145ba8..39307fd 100644 --- a/webapp/src/app/components/shico/trackerGraphs.directive.js +++ b/webapp/src/app/components/shico/trackerGraphs.directive.js @@ -8,7 +8,7 @@ function trackerGraphs() { var directive = { scope: {}, // Directive has it's own personal scope - templateUrl: '/app/components/shico/trackerGraphs.template.html', + templateUrl: 'app/components/shico/trackerGraphs.template.html', controllerAs: 'vm', controller: 'TrackerGraphsController' }; diff --git a/webapp/src/app/components/shico/trackerGraphs.template.html b/webapp/src/app/components/shico/trackerGraphs.template.html index 0e7cfac..ddb8907 100644 --- a/webapp/src/app/components/shico/trackerGraphs.template.html +++ b/webapp/src/app/components/shico/trackerGraphs.template.html @@ -6,7 +6,7 @@
-
+
{{ vm.getYearLabel(yearIdx) }}
- - + +
- + @@ -44,18 +47,9 @@
-
- -
-
- -
+
diff --git a/webapp/src/app/index.module.js b/webapp/src/app/index.module.js index a0bf628..08630b1 100644 --- a/webapp/src/app/index.module.js +++ b/webapp/src/app/index.module.js @@ -11,6 +11,7 @@ 'rzModule', 'cgBusy', 'ngSanitize', - 'ngCsv' + 'ngCsv', + 'hc.marked' ]); })(); diff --git a/webapp/src/help/algorithm.md b/webapp/src/help/algorithm.md new file mode 100644 index 0000000..aac182f --- /dev/null +++ b/webapp/src/help/algorithm.md @@ -0,0 +1,6 @@ +### Algorithm + +The vocabulary monitor contains can use two different algorithms for generating vocabularies. These control which words are used as seed terms for each model: + + - The non-adaptive vocabulary generator uses the same seed terms each time to generate the related terms. + - The adaptive vocabulary generator uses the related terms generated by one semantic model as seed terms for the next semantic model. This adds an additional possibility: it allows for the semantic models to be used in chronological order, or in reverse chronological order -- searching forwards or backwards in time.