Skip to content

GitLab

Commit 7098a21f authored by Dimitrios Papadopoulos's avatar Dimitrios Papadopoulos
Browse files

Updated comments

parent 95780ca8
Hide whitespace changes
Inline Side-by-side
src/DocumentIngestion.ipynb
View file @ 7098a21f
%% Cell type:markdown id: tags:
---
#### Python Notebook to read all the text files from a text dataset into Neo4j database (syntactilally-based, SVO)
---
1. List all the text files in the sub-directories your dataset.
2. Read all the files.
3. Create nodes, where n(nodes) = n(files)
4. Dump the text files into individual nodes where every node is a document using Graphaware's NLP pipeline.
5. Sample scripts for entity extraction
---
%% Cell type:markdown id: tags:
Imports<br>glob --> for iterating through the folders and sub-folders
%% Cell type:code id: tags:
``` python
import glob
import csv
```
%% Cell type:markdown id: tags:
Specifying the path for the files, the wildcards at the end of the path denote that all the files from all the subdirectories from the bbc folder will be accessed.
%% Cell type:code id: tags:
``` python
folder_path = '/home/earendil/Desktop/ML_playground/Neo4j-NLP/data/biomarker/*'
```
%% Cell type:markdown id: tags:
glob will help iterate through the entire folder path, allowing wildcards
%% Cell type:code id: tags:
``` python
from py2neo import *
```
%% Cell type:code id: tags:
``` python
from neo4j import GraphDatabase
uri = "bolt://localhost:7687"
driver = GraphDatabase.driver(uri, auth=("neo4j", "1qazxsw2"))
```
%% Cell type:code id: tags:
``` python
authenticate("localhost:7474", "neo4j", "1qazxsw2")
graph = Graph("http://localhost:7474/db/data/")
#graph = Graph(host='localhost', user='neo4j',password='password')
tx = graph.begin()
```
%% Cell type:code id: tags:
``` python
for filename in glob.glob(folder_path):
with open(filename, 'r') as f:
file_contents = f.read()
Nodes = Node("Article",Text=str(file_contents),path=filename)
print(Nodes)
graph.create(Nodes)
tx.merge(Nodes)
```
%% Cell type:markdown id: tags:
CALL ga.nlp.processor.addPipeline({
name:"pipeline",
textProcessor: 'com.graphaware.nlp.processor.stanford.ee.processor.EnterpriseStanfordTextProcessor',
processingSteps: {tokenize:true, ner:true, dependencies:true, relations:true, open:true, sentiment:true}
})
%% Cell type:code id: tags:
``` python
#NEO4J scripts (non-Python code)
CALL ga.nlp.processor.addPipeline({
name: 'pipeline2',
textProcessor: 'com.graphaware.nlp.processor.stanford.StanfordTextProcessor',
processingSteps: {tokenizerAndSentiment:true, ner: true, dependency: true}})
```
%% Cell type:code id: tags:
``` python
#NEO4J scripts (non-Python code)
CALL apoc.periodic.iterate(
'MATCH (n:Article) RETURN n',
'CALL ga.nlp.annotate({
text: n.Text,
id: id(n),
pipeline: "pipeline2",
checkLanguage:false
})
YIELD result MERGE (n)-[:HAS_ANNOTATED_TEXT]->(result)',
{batchSize:1, iterateList:false})
```
%% Cell type:code id: tags:
``` python
#NEO4J scripts (non-Python code)
MATCH (s:TagOccurrence)<-[]-(a:Sentence)-[]->(v:TagOccurrence),
(a)-[]->(o:TagOccurrence)
WHERE s.pos IN [['NNP']] AND v.pos IN [['VBZ']] AND o.pos IN [['NN']]
RETURN DISTINCT s.value, v.value, o.value
```
%% Cell type:code id: tags:
``` python
#NEO4J scripts (non-Python code)
MATCH (s:TagOccurrence)<-[]-(a:Sentence)-[]->(v:TagOccurrence),
(a)-[]->(o:TagOccurrence)
WHERE s.pos IN [['NNP']] AND v.pos IN [['VBZ']] AND o.pos IN [['NN']] AND abs(v.startPosition-s.endPosition)<10 AND abs(o.startPosition-v.endPosition)<10
RETURN DISTINCT s.value, v.value, o.value, v.startPosition-s.endPosition
```
%% Cell type:code id: tags:
``` python
#NEO4J scripts (non-Python code)
MATCH p= (ar:Article)-[:HAS_ANNOTATED_TEXT]->(an:AnnotatedText)-[:CONTAINS_SENTENCE]->(se:Sentence)-[:SENTENCE_TAG_OCCURRENCE]-(s:TagOccurrence)-[:NSUBJ]-(v:TagOccurrence)-[:DOBJ]-(o:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:COMPOUND]-(co:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:AMOD]-(am:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:NMOD]-(nm:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:NMOD]-(nm:TagOccurrence)
OPTIONAL MATCH (nm:TagOccurrence)-[:APPOS]-(apr:TagOccurrence)
RETURN se.text as Text, s.value as Subject, v.value as Predicate, am.value as Desc1, nm.value as Desc2,co.value as Desc3, apr.value as Prop, o.value as Object LIMIT 200
```
%% Cell type:code id: tags:
``` python
MATCH p= (ar:Article)-[:HAS_ANNOTATED_TEXT]->(an:AnnotatedText)-[:CONTAINS_SENTENCE]->(se:Sentence)-[:SENTENCE_TAG_OCCURRENCE]-(s:TagOccurrence)-[:NSUBJ]-(v:TagOccurrence)-[:DOBJ]-(o:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:COMPOUND]-(co:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:AMOD]-(am:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:NMOD]-(nm:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:NMOD]-(nm:TagOccurrence)
OPTIONAL MATCH (nm:TagOccurrence)-[:APPOS]-(apr:TagOccurrence)
CALL apoc.create.relationship(s, toString(v.value), {}, o) YIELD rel
RETURN se.text as Text, s.value as Subject, v.value as Predicate, am.value as Desc1, nm.value as Desc2,co.value as Desc3, apr.value as Prop, o.value as Object LIMIT 200
```
%% Cell type:code id: tags:
``` python
MATCH a = ()-[:explore]-()
MATCH b = ()-[:combine]-()
MATCH c = ()-[:discover]-()
MATCH d = ()-[:changed]-()
MATCH e = ()-[:link]-()
RETURN a,b,c,d,e
```
%% Cell type:code id: tags:
``` python
MATCH p= (ar:Article)-[:HAS_ANNOTATED_TEXT]->(an:AnnotatedText)-[:CONTAINS_SENTENCE]->(se:Sentence)-[:SENTENCE_TAG_OCCURRENCE]-(s:TagOccurrence)-[:NSUBJ]-(v:TagOccurrence)-[:DOBJ]-(o:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:AMOD]-(am:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:NMOD]-(nm:TagOccurrence)
OPTIONAL MATCH (nm:TagOccurrence)-[:APPOS]-(apr:TagOccurrence)
OPTIONAL MATCH (s:TagOccurrence)-[:COMPOUND]-(coms:TagOccurrence)
OPTIONAL MATCH (o:TagOccurrence)-[:COMPOUND]-(como:TagOccurrence)
RETURN se.id as SentenceID,
toString(COALESCE(coms.value+' ', ' '))+
toString(s.value) as Subject,
v.value as Predicate,
toString(COALESCE(am.value+' ',' '))+
toString(COALESCE(nm.value+' ', ' '))+
toString(COALESCE(apr.value+' ',' '))+
toString(COALESCE(como.value+' ',' '))+
toString(o.value) as Object
LIMIT 200
```
%% Cell type:markdown id: tags:
#### --------------
%% Cell type:markdown id: tags:
---
#### OpenIE for automated triple extraction based on open information extraction triples
---
1. OpenIE will read a file and extract triples using Stanford Core NLP
0. Coref resolution reads a file and substitutes the antecedents in place, producing a resolved text.
1. OpenIE will read the text and extract triples using Stanford Core NLP
2. The output is a list of dicts containing "subject-relation-object" triples.
3. Transfer these triples to Neo4j as nodes-edges.
4. Due to the OIE implementation, similar nodes and edges (almost identical meanings) are present, we can remove them see below, but we may also miss information this way.
3. Due to the OIE implementation, similar nodes and edges (almost identical meanings) are present, we can remove them see below, but we may also miss information this way.
4. Transfer these triples to Neo4j as nodes-edges.
---
%% Cell type:code id: tags:
``` python
from openie import StanfordOpenIE
with StanfordOpenIE() as client:
with open('/home/earendil/Desktop/ML_playground/Neo4j-NLP/data/siris/tresspass.txt', 'r', encoding='utf8') as r:
corpus = r.read().replace('\n', ' ').replace('\r', '')
triples_corpus = client.annotate(corpus,
properties={
"annotators":"tokenize,ssplit,pos,lemma,depparse,ner,coref,mention,natlog,openie",
"outputFormat": "json",
"openie.format": "ollie",
"openie.resolve_coref": "false",
})
print('Corpus: %s [...].' % corpus[0:80])
print('Found %s triples in the corpus.' % len(triples_corpus))
for triple in triples_corpus:
print('|-', triple)
```
%%%% Output: stream
Starting server with command: java -Xmx8G -cp /home/earendil/stanfordnlp_resources/stanford-corenlp-full-2018-10-05/* edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 60000 -threads 5 -maxCharLength 100000 -quiet True -serverProperties corenlp_server-770b57d3fc6b4732.props -preload openie
Corpus: TRESSPASS: robusT Risk basEd Screening and alert System for PASSengers and lugga [...].
Found 22 triples in the corpus.
|- {'subject': 'TRESSPASS project', 'relation': 'includes', 'object': 'air border points'}
|- {'subject': 'TRESSPASS project', 'relation': 'includes', 'object': 'air border crossing points'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats of refugees'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats on Mediterranean'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats of refugees on Mediterranean'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats of refugees on Mediterranean'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats of refugees'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'border crossings'}
|- {'subject': 'It', 'relation': 'excludes border crossings outside of', 'object': 'border'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats on Mediterranean'}
|- {'subject': 'this', 'relation': 'regards to', 'object': 'threats'}
|- {'subject': 'other threats', 'relation': 'posed by', 'object': 'state-actors'}
|- {'subject': 'threats', 'relation': 'posed by', 'object': 'state-actors'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'other threats posed by state-actors'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'threats posed'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'other threats posed'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'threats posed by state-actors'}
|- {'subject': 'other known risk-based border management projects', 'relation': 'is in', 'object': 'describe'}
|- {'subject': 'TRESSPASS', 'relation': 'will develop', 'object': 'border management concept'}
%% Cell type:code id: tags:
``` python
#Neural coreference resolution https://stackoverflow.com/questions/50004797/anaphora-resolution-in-stanford-nlp-using-python
#NEEDS SPACY 2.0.13
#Don't run this, run the AllenNLP script below instead
import spacy
nlp = spacy.load('en_coref_md')
doc = nlp(corpus)
corpus_coref = doc._.coref_resolved
triples_corpus = client.annotate(corpus_coref,
properties={
"annotators":"tokenize,ssplit,pos,lemma,depparse,ner,coref,mention,natlog,openie",
"outputFormat": "json",
"openie.format": "ollie",
"openie.resolve_coref": "false",
})
print('Corpus: %s [...].' % corpus_coref[0:80])
print('Found %s triples in the corpus.' % len(triples_corpus))
for triple in triples_corpus:
print('|-', triple)
```
%%%% Output: error
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-6-37f990d4718a> in <module>
1 #Neural coreference resolution https://stackoverflow.com/questions/50004797/anaphora-resolution-in-stanford-nlp-using-python
2 import spacy
----> 3 nlp = spacy.load('en_coref_md')
4 doc = nlp(corpus)
5 corpus_coref = doc._.coref_resolved
~/anaconda3/envs/athnlp/lib/python3.6/site-packages/spacy/__init__.py in load(name, **overrides)
25 if depr_path not in (True, False, None):
26 deprecation_warning(Warnings.W001.format(path=depr_path))
---> 27 return util.load_model(name, **overrides)
28
29
~/anaconda3/envs/athnlp/lib/python3.6/site-packages/spacy/util.py in load_model(name, **overrides)
164 return load_model_from_link(name, **overrides)
165 if is_package(name): # installed as package
--> 166 return load_model_from_package(name, **overrides)
167 if Path(name).exists(): # path to model data directory
168 return load_model_from_path(Path(name), **overrides)
~/anaconda3/envs/athnlp/lib/python3.6/site-packages/spacy/util.py in load_model_from_package(name, **overrides)
184 def load_model_from_package(name, **overrides):
185 """Load a model from an installed package."""
--> 186 cls = importlib.import_module(name)
187 return cls.load(**overrides)
188
~/anaconda3/envs/athnlp/lib/python3.6/importlib/__init__.py in import_module(name, package)
124 break
125 level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)
127
128
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap.py in _gcd_import(name, package, level)
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap.py in _find_and_load(name, import_)
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)
~/anaconda3/envs/athnlp/lib/python3.6/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)
~/anaconda3/envs/athnlp/lib/python3.6/site-packages/en_coref_md/__init__.py in <module>
4 from pathlib import Path
5 from spacy.util import load_model_from_init_py, get_model_meta
----> 6 from en_coref_md.neuralcoref import NeuralCoref
7
8 __version__ = get_model_meta(Path(__file__).parent)['version']
~/anaconda3/envs/athnlp/lib/python3.6/site-packages/en_coref_md/neuralcoref/__init__.py in <module>
----> 1 from .neuralcoref import NeuralCoref
cymem.pxd in init en_coref_md.neuralcoref.neuralcoref()
ValueError: cymem.cymem.Pool has the wrong size, try recompiling. Expected 64, got 48
%% Cell type:code id: tags:
``` python
# Neural Coref using Allennlp
#NEEDS SPACY 2.2.0
from allennlp.predictors import CorefPredictor
predictor = CorefPredictor.from_path("https://s3-us-west-2.amazonaws.com/allennlp/models/coref-model-2018.02.05.tar.gz")
corpus_coref = predictor.coref_resolved(document=corpus)
triples_corpus = client.annotate(corpus_coref,
properties={
"annotators":"tokenize,ssplit,pos,lemma,depparse,ner,coref,mention,natlog,openie",
"outputFormat": "json",
"openie.format": "ollie",
"openie.resolve_coref": "false",
})
print('Corpus: %s [...].' % corpus_coref[0:80])
print('Found %s triples in the corpus.' % len(triples_corpus))
for triple in triples_corpus:
print('|-', triple)
```
%%%% Output: stream
/home/earendil/anaconda3/envs/athnlp/lib/python3.6/site-packages/torch/nn/modules/rnn.py:51: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
"num_layers={}".format(dropout, num_layers))
/home/earendil/anaconda3/envs/athnlp/lib/python3.6/site-packages/allennlp/data/token_indexers/token_characters_indexer.py:56: UserWarning: You are using the default value (0) of `min_padding_length`, which can cause some subtle bugs (more info see https://github.com/allenai/allennlp/issues/1954). Strongly recommend to set a value, usually the maximum size of the convolutional layer size when using CnnEncoder.
UserWarning)
/tmp/pip-req-build-ocx5vxk7/aten/src/ATen/native/LegacyDefinitions.cpp:19: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead.
WARNING:allennlp.models.model:Encountered the antecedent_indices key in the model's return dictionary which couldn't be split by the batch size. Key will be ignored.
%%%% Output: stream
Corpus: TRESSPASS: robusT Risk basEd Screening and alert System for PASSengers and lugga [...].
Found 22 triples in the corpus.
|- {'subject': 'TRESSPASS project', 'relation': 'includes', 'object': 'air border points'}
|- {'subject': 'TRESSPASS project', 'relation': 'includes', 'object': 'air border crossing points'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats of refugees'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats on Mediterranean'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats of refugees on Mediterranean'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats of refugees on Mediterranean'}
|- {'subject': 'It', 'relation': 'happens with', 'object': 'boats of refugees'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'border crossings'}
|- {'subject': 'It', 'relation': 'excludes border crossings outside of', 'object': 'border'}
|- {'subject': 'It', 'relation': 'excludes border crossings', 'object': 'happens with boats on Mediterranean'}
|- {'subject': 'this', 'relation': 'regards to', 'object': 'threats'}
|- {'subject': 'other threats', 'relation': 'posed by', 'object': 'state-actors'}
|- {'subject': 'threats', 'relation': 'posed by', 'object': 'state-actors'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'other threats posed by state-actors'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'threats posed'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'other threats posed'}
|- {'subject': 'It', 'relation': 'excludes', 'object': 'threats posed by state-actors'}
|- {'subject': 'other known risk-based border management projects', 'relation': 'is in', 'object': 'describe'}
|- {'subject': 'TRESSPASS', 'relation': 'will develop', 'object': 'border management concept'}
%% Cell type:code id: tags:
``` python
#remove duplicate triples (i.e. where some objects are subsets of others )
#MUST BE RUN, but not currently used. To use it substitute triples with unique_triples on the code blocks
triples = sorted(triples_corpus, key = lambda i: len(i['object']),reverse=True)
duplicates= []
unique_triples = []
c=0
for d in triples:
t = tuple(d.items())
if (triples[c]["relation"]) not in duplicates:
duplicates.append(triples[c]["relation"])
unique_triples.append(d)
c+=1
print("===============")
print('Number of triples generated by OpenIE:',len(triples))
print('Number of triples after removing duplicates:',len(unique_triples))
```
%%%% Output: stream
===============
Number of triples generated by OpenIE: 22
Number of triples after removing duplicates: 9
%% Cell type:code id: tags:
``` python
#connect to Neo4j DB
from py2neo import *
from neo4j import GraphDatabase
uri = "bolt://localhost:7687"
authenticate("localhost:7474", "neo4j", "1qazxsw2")
driver = GraphDatabase.driver(uri, auth=("neo4j", "1qazxsw2"))
graph = Graph("http://localhost:7474/db/data/")
```
%% Cell type:code id: tags:
``` python
# create nodes and relations from extracted triples
tx = graph.begin()
for i in range(len(triples)):
u1 = Node("Subject", name=triples[i]['subject'], corpus='CORDIS', entry='TRESSPASS')
u2 = Node("Object", name=triples[i]['object'],corpus='CORDIS', entry='TRESSPASS')
graph.merge(u1 | u2)
relation = Relationship(u1,triples[i]['relation'], u2)
graph.merge(relation)
```
%% Cell type:code id: tags:
``` python
# create nodes for the Corpus and the Entry if it doesnt exist already (merge instead of create)
u1 = Node("Entry", name='TRESSPASS', corpus='CORDIS')
u2 = Node("Corpus", name='CORDIS')
graph.merge(u1 | u2)
relation = Relationship(u2,'has_entry', u1)
graph.merge(relation)
```
%% Cell type:code id: tags:
``` python
#Gather all the triples under the same node
tx.run("MATCH (sub:Subject), (p:Entry {name: 'TRESSPASS' }) WHERE sub.entry IN ['TRESSPASS'] WITH p, COLLECT(sub) AS subs CREATE (p)-[:contains]->(c: Combo {name:'Triples-TRESSPASS'}) FOREACH(s IN subs | CREATE (c)-[:has_triple]->(s)) RETURN *")
tx.process()
tx.commit()
```
%% Cell type:markdown id: tags:
===================================================================
Misc scripts
---
#### Misc individual scripts for coref resolution (used above)
---
%% Cell type:code id: tags:
``` python
#Neural coreference resolution https://stackoverflow.com/questions/50004797/anaphora-resolution-in-stanford-nlp-using-python
import spacy
nlp = spacy.load('en_coref_md')
doc = nlp(corpus)
print(doc._.coref_clusters)
print(doc._.coref_resolved)
```
%% Cell type:code id: tags:
``` python
#Co-reference resolution using AthNLP
from allennlp.predictors import CorefPredictor
predictor = CorefPredictor.from_path("https://s3-us-west-2.amazonaws.com/allennlp/models/coref-model-2018.02.05.tar.gz")
print(predictor.coref_resolved(document=corpus))
```
%% Cell type:code id: tags:
``` python
#Co-reference resolution using StanfordCoreNLP (TODO)
def resolve(corenlp_output):
""" Transfer the word form of the antecedent to its associated pronominal anaphor(s) """
for coref in corenlp_output['corefs']:
mentions = corenlp_output['corefs'][coref]
antecedent = mentions[0] # the antecedent is the first mention in the coreference chain
for j in range(1, len(mentions)):
mention = mentions[j]
if mention['type'] == 'PRONOMINAL':
# get the attributes of the target mention in the corresponding sentence
target_sentence = mention['sentNum']
target_token = mention['startIndex'] - 1
# transfer the antecedent's word form to the appropriate token in the sentence
corenlp_output['sentences'][target_sentence - 1]['tokens'][target_token]['word'] = antecedent['text']
def print_resolved(corenlp_output):
""" Print the "resolved" output """
possessives = ['hers', 'his', 'their', 'theirs']
for sentence in corenlp_output['sentences']:
for token in sentence['tokens']:
output_word = token['word']
# check lemmas as well as tags for possessive pronouns in case of tagging errors
if token['lemma'] in possessives or token['pos'] == 'PRP$':
output_word += "'s" # add the possessive morpheme
output_word += token['after']
print(output_word, end='')
output = client.annotate(corpus, properties= {'annotators':'dcoref','outputFormat':'json','ner.useSUTime':'false'})
print(output)
resolve(output)
print('Original:', text)
print('Resolved: ', end='')
print_resolved(output)
```
%% Cell type:code id: tags:
``` python
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment