166 questions
0
votes
0
answers
35
views
Is in TorchSharp implementation of TorchText completed? If not, what are alternatives?
For TorchSharp text classification example there is TorchText used to load data set.
I am not sure what I am doing wrong, but I can not find any usings to import this library.
For TorchSharp MNIST ...
- 631
0
votes
0
answers
39
views
Getting import error from import `to_map_style_dataset`
I'm just started studying the annotated transformer. And I got an error with importing.
The error indicates importing:
from torchtext.data.functional import to_map_style_dataset
has this info:
-------...
- 37
1
vote
2
answers
455
views
`AttributeError` when importing `torchtext` - "Symbol not found in libc10.dylib"
I'm trying to import torchtext in my Python script, but I'm encountering an error related to a missing symbol in libc10.dylib. When I run my script, I get the following error:
-------------------------...
- 11
0
votes
2
answers
683
views
import torchtext - OSError: [WinError 127]
I'm trying to run a Test.py with 'import torchtext' in it.
it throws this error:
Traceback (most recent call last): File
"C:\Users\anryu\pythonProject\Test.py", line 10, in <module>...
- 21
1
vote
2
answers
1k
views
"OSError: [WinError 127] The specified procedure could not be found" when importing torchtext
When importing get_tokenizer from torchtext:
from torchtext.data.utils import get_tokenizer
it throws this error:
Traceback (most recent call last):
File "D:\PythonProjects\FakeNews\train.py&...
2
votes
1
answer
1k
views
ImportError: cannot import name 'DILL_AVAILABLE'
I want to work with IMDB datasets. Trying to load using following command:
from torchtext.datasets import IMDB
train_iter = IMDB(root='~/datasets', split='train')
I am getting following error:
...
0
votes
0
answers
96
views
Getting torchtext.legacy module not found error
I have found a Language Detection model in github and I want to try it. It was written using torchtext 0.11. When I try to run it with torchtext 0.18 installed, I get torchtext.legacy module not found ...
- 3
0
votes
0
answers
58
views
torchtext and fasttext vectorization
I found example with use ready vectors for Glove (for FastText the same example) :
`# set up fields
TEXT = data.Field(lower=True, include_lengths=True, batch_first=True)
LABEL = data.Field(sequential=...
- 11
0
votes
1
answer
30
views
Subsampling when training word embeddings
NLP newbie here with a question about word embeddings. As a learning exercise, I'm trying to train my own set of word embeddings based on word2vec. I have a corpus of english sentences that I've ...
- 5,730
1
vote
1
answer
125
views
Torchtext functions in Newest version analogue
Good day all, i'm trying to solve task, where it was used previously torchtext.dataset.TranslationDataset, torch.data.Field and torch.data.BucketIterator. But, after updating they were removed and i ...
- 13
0
votes
0
answers
53
views
How to pad a text with custom length after build the vocab in pytorch
I used torchtext vocab to convert the text to index
For example 1 have 2 names
aaban aabharan
After vocab:
[0, 0, 1, 0, 2] [0, 0, 1, 3, 0, 4, 0, 2]
Since the length of longest name in my data is 24
...
0
votes
0
answers
13
views
output shape of BucketIterator
enter image description here
I'm trying to create a batch through BucketIterator from train_data and test_data created by TabularDataset.splits().
I wanted a batch of the shape (128,1), but it returns ...
- 3
1
vote
1
answer
440
views
AttributeError: module 'torchtext.data' has no attribute 'Field' not fixed with torchtext.legacy
I am trying to use torchtext.data.Field() from torchtext but it is not working in any way I have tried.
I have tried with
from torchtext.legacy import data but it does not seem to work. I have also ...
- 13
2
votes
1
answer
480
views
Fail to import torchtext KeyError: 'SP_DIR'
I failed to import torchtext with the following error. I tried it with a fresh conda env install (under a different python version) and still got the same issue.
Originally I was able to use torchtext ...
- 825
0
votes
1
answer
683
views
Why am I getting an error while loading IMDB Dataset
from torchtext.datasets import WikiText2, IMDB
from torchtext.data.utils import get_tokenizer
from torchtext.vocab import build_vocab_from_iterator
tkzer = get_tokenizer('basic_english')
tr_iter = ...
- 2,250
0
votes
2
answers
388
views
Updating old code with new torch vocab methods (stoi and itos methods changed)
I am trying to create a Japanese-English translation model following this Medium article.
https://arusl.medium.com/japanese-english-language-translation-with-transformer-using-pytorch-243738146806
...
- 15
0
votes
1
answer
844
views
Pytorch T5 training loss not changing
I am trying to fine tune a T5 model for more accurate summarization, but my loss is very high and does not change with each epoch. I have tried increasing the learning rate, but the model still does ...
- 49
3
votes
2
answers
5k
views
Question in Pytorch transformer_tutorial about 'NoneType' object has no attribute 'Lock'
When I run the transformer_tutorial code from Pytorch (https://pytorch.org/tutorials/beginner/transformer_tutorial.html), I meet a problem in build_vocab_from_iterator.
from torchtext.datasets import ...
- 31
1
vote
0
answers
126
views
sparse gradient in text classification with pytorch
I am trying to train a text classification module.
When I use Adam, RAdam, or RMSProp for my optimizer, I get the following error:
RuntimeError: Adam/RAdam/RMSProp does not support sparse gradients
...
- 175
0
votes
1
answer
713
views
Module "torchtext.data" has no attribute "Field"
I am trying to define a vocabulary with PyTorch to later input my dataset into a Neural Network. This would be the first step to create a dataset.
This is the code I am using:
text_field = torchtext....
- 1
1
vote
1
answer
8k
views
Install torchtext with pytorch 1.13.1 with cuda 11.7
I currently have PyTorch 1.13.1 with CUDA 11.7 installed and it's working perfectly fine. However, when I try to install TorchText using 'pip install torchtext', it automatically installs PyTorch 2 ...
- 157
0
votes
1
answer
339
views
cannot import name 'unicode_csv_reader' from 'torchtext.utils'
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-25-b2b2b2ad7bcb> in <...
0
votes
0
answers
62
views
How to install torchtext in cmake project on windows 10?
How to install torchtext in cmake project on windows 10?
Error:
CMake Error at X:\Programming\C++\Tests\TestPyTorch1\CMakeLists.txt:11 (find_package):
By not providing "FindTorchText.cmake"...
2
votes
1
answer
2k
views
Error: AttributeError: module 'torchtext.data' has no attribute 'Field'
Actually I saw couple of same questions in the forum but i still don't have the right solution.
here's my code block:
import torchtext as data
import torchtext.datasets as datasets
...
...
text_field ...
- 77
2
votes
0
answers
138
views
How to create a custom parallel corpus for machine translation with recent versions of pytorch and torchtext?
I am trying to train a model for NMT on a custom dataset. I found this great tutorial on youtube along with the accompanying repo, but it uses an old version of PyTorch and torchtext. More recent ...
- 43
1
vote
1
answer
300
views
How to bring pytorch datasets into pandas dataframe
I have seen a lot of code on how to convert pandas data to pytorch dataset. However, I haven't found or been able to figure out to do the reverse. i.e. Load pytorch dataset into pandas dataframe. I ...
- 2,054
2
votes
1
answer
831
views
ImportError: cannot import name '_log_class_usage' from 'torchtext.utils'
I recently updated pytorch and torchtext from anaconda, in order to run this tutorial from the torchtext website, but it seems that my installation of torchtext has been broken. Everytime that I try ...
- 4,368
1
vote
0
answers
175
views
How to load Field from torchtext.data?
I have installed torchtext v 0.14.1. I am on Windows. I tried to load Field but I got the following error. Can anybody help me fix it?
from torchtext.data import Field
ImportError: cannot import name ...
- 309
0
votes
0
answers
2k
views
ModuleNotFoundError: No module named 'torchdata'
How can get the training data as text (or list of texts) from PyTorch Dataset(<torchtext.data.dataset.Dataset>) Object ? Or more specifically from torchtext squad 1 or 2 the Q&a version
I ...
1
vote
1
answer
1k
views
Torchtext vocab getting tokens from index
from torchtext.vocab import Vocab
from collections import Counter
def create_vocab(file_, tokenizer):
counter_dict = Counter()
for sentence in file_:
counter_dict.update(tokenizer(...
- 386
0
votes
1
answer
1k
views
module ‘torchtext.data‘ has no attribute ‘LabelField‘
I am trying to import data like this:
from torchtext import data
When I am trying to use data.LabelField like this:
LABEL = data.LabelField(dtype = torch.float)
It is showing this error:
module ‘...
- 178
0
votes
1
answer
74
views
What do I do wrong when install OpenKiwi?
I tried to work with OpenKiwi to Anaconda3
and after installation (pip install openkiwi)
I execute following code
(I do this because I want to create openkiwi vocabulary) :
import warnings
from ...
- 3
0
votes
1
answer
802
views
Error when importing torchtext.data.datasets_utils
Im trying to import torchtext but got errors even after trying with different version.
from torch.utils.data import DataLoader
from torch.nn.utils.rnn import pad_sequence
import math
from torch.nn ...
- 31
0
votes
0
answers
232
views
'Vocab' object has no attribute 'GloVe'
def load_pretrained_embedding(words, pretrained_vocab_path=None, emb_size=100, type="glove"):
embed = torch.normal(mean=0, std=1, size=(len(words), emb_size))
if type == "glove&...
0
votes
1
answer
268
views
How to add modules from torchtext.data?
I'm trying to do this from torchtext.data import TabularDataset, Field, LabelField, BucketIterator, however, I get the following exception:
ImportError: cannot import name 'TabularDataset' from '...
- 1,956
2
votes
1
answer
840
views
how to pad a text after build the vocab in pytorch
I used torchtext vocab to convert the text to index, but which function should I use to make all the index list be the same length before I send them to the net?
For example I have 2 texts:
I am a ...
- 67
4
votes
2
answers
5k
views
Unable to build vocab for a torchtext text classification
I'm trying to prepare a custom dataset loaded from a csv file in order to use in a torchtext text binary classification problem. It's a basic dataset with news headlines and a market sentiment label ...
- 53
1
vote
0
answers
561
views
Torchtext import error on macbook air m1. Used pip3 to install all of it
While the installation went smooth, I keep facing this error when I try to import torchtext on my macbook air m1
---------------------------------------------------------------------------
OSError ...
2
votes
1
answer
5k
views
ImportError: cannot import name 'unicode_csv_reader' from 'torchtext.utils'
I am trying to import these libraries - data, Field, LabelField, TabularDataset, BucketIterator from torchtext but I have this error. Could anyone suggest how to solve this problem please? Thank you ...
- 21
0
votes
1
answer
310
views
PyTorch: How to convert SentencePiece numbers into tokens
PyTorch's torchtext can convert tokens into integers via sentencepiece-numericalizer. E.g. "is" -> 17".
What about the inverse operation, from integer to a token? E.g. 17 --> &...
- 9,139
0
votes
1
answer
949
views
How to create an iterable DataPipe with PyTorch using txt files
I have two text files to train a transformer model. However, instead of using PyTorch's own datasets, I'm using something I downloaded from the internet.
source = open('./train_de.de', encoding='utf-8'...
- 3
1
vote
0
answers
67
views
Preventing data persistence in pickle
I am trying to read in a pickle file containing a torchtext.data.field.Field object, however my current repo is using a more updated version of torchtext and thus the data type is torchtext.legacy....
- 473
0
votes
1
answer
509
views
Missmatching target size in criterion
I'm trying to use pytorch in for the IMBD dataset, to predict the positive and negative reviews. When I get to the training state, the following error is given by the criterion function:
ValueError: ...
- 500
2
votes
1
answer
1k
views
How to load data from csv with torchtext 0.12.0
I am trying to figure out the new method to load a csv dataset for Torchtext, but I can't find any clear explanations.
Given a csv with two columns: text, class, all of the tutorials I can find for ...
- 1,196
0
votes
1
answer
7k
views
Cannot import name 'functional_datapipe' from 'torch.utils.data'
When I am running datasets_utils.py from '/usr/local/lib/python3.7/dist-packages/torchtext/data/datasets_utils.py' in Google Colab, the following error occurs even with the most updated versions of ...
- 103
5
votes
2
answers
8k
views
AttributeError: module 'torchtext' has no attribute 'legacy'
I am trying to use torchtext to process test data, however, I get the error: "AttributeError: module 'torchtext' has no attribute 'legacy'", when I run the following code. Can anyone please ...
- 51
5
votes
3
answers
6k
views
TorchText Vocab TypeError: Vocab.__init__() got an unexpected keyword argument 'min_freq'
I am working on a CNN Sentiment analysis machine learning model which uses the IMDb dataset provided by the Torchtext library.
On one of my lines of code
vocab = Vocab(counter, min_freq = 1, specials=(...
- 53
6
votes
1
answer
7k
views
can't import torchtext.legacy.data
as i know, from torchtext 0.9.0, torchtext.data and torchtext.dataset are moved to torchtext.legacy
but my 0.12.0 torchtext can't import torchtext.legacy
while it can import torchtext.data
I tried if ...
- 107
0
votes
1
answer
227
views
FileNotFoundError: Entity folder does not exist! in Google Colab
Can anyone help me in sorting out this issue?
When I run these lines in Colab
:param files_name: containing training and validation samples list file.
:param boxes_and_transcripts_folder: gt or ocr ...
- 44
0
votes
1
answer
286
views
Using torchtext vocab with torchscript
I'm trying to use the torchtext vocab layer along with torchscript but I'm getting some errors and I was wondering if someone here has made it work.
My current model is
class VocabText(torch.nn.Module)...
- 816