All Questions
375 questions
0
votes
1
answer
59
views
Why is Keras pretrained BERT MaskedLM producing inconsistent predictions?
I am trying to use keras-nlp with a pretrained masked BERT model to predict some tokens in a sequence. However the model produces inconsistent results. What could be wrong or am i misunderstanding ...
- 67
0
votes
0
answers
88
views
"KeyError: 0" when calling model.fit() in Keras
I am training a simple sequential model for NLP in keras. Here is the model architecture:
from keras.models import Sequential
from keras import layers
from keras.layers import Embedding, Flatten, ...
- 113
0
votes
0
answers
39
views
why isn't tf.keras.layers.TextVectorization accepting standardization=None?
I'm still trying to get this work (and to learn!) so I am using a tiny corpus.
I do some preprocessing on the text in order to get specific bi-gram collocations using nltk (not relevant here but I ...
- 131
0
votes
1
answer
279
views
save and load keras transformer model
i am still new to deep learning right now i follow this keras tutorial to make an translation model using transformer here the link. Everything works just fine but i have no idea how to save the model,...
0
votes
1
answer
91
views
Preprocessing a large dataset with tf.layers.TextVectorization gives Memory Errors
I have around 300k files, which is around 9GB of medical literature.
My goal is to establish the frequency of ALL tokens in the dataset and serialize them to a csv file (token, frequency).
To achieve ...
0
votes
1
answer
692
views
Error.__init__() got an unexpected keyword argument 'trainable' when try to load a .keras model
I've trained a NER model and save it as .keras file, when I try to load the model, it pops up this error
TypeError: <class 'modeling.NERModel'> could not be deserialized properly. Please ensure ...
- 3
1
vote
0
answers
54
views
ResourceExhaustedError In Tensorflow BERT Classifier
I am trying to use the BertClassifier from the keras_nlp library but when I train the model I get this error:
2024-03-22 22:53:03.932926: W external/local_tsl/tsl/framework/bfc_allocator.cc:487] ...
- 41
0
votes
1
answer
74
views
Error when I trying to run a trained ner model on local pc
import re
import pickle
import keras
import tensorflow as tf
from keras.models import Sequential
from keras.layers import TFSMLayer
import numpy as np
class CustomNonPaddingTokenLoss(keras.losses....
- 3
2
votes
1
answer
4k
views
AttributeError: module 'keras.src.backend' has no attribute 'floatx' When compiling a neural network for sentiment analysis
from tensorflow.keras.models import Sequential
from tensorflow.keras import layers
# set the embedding dimension
embedding_dim = 100
# create the model
model = Sequential([
layers.Embedding(...
- 71
0
votes
1
answer
98
views
Keras ValueError: cannot reshape array of size
I'm facing an error which I can't understand using Keras for a prediction task.
Here is my code:
import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from keras.models ...
1
vote
2
answers
398
views
How to save Keras TextVectorization layer configuration with custom standardization function into a pickle file and reload it?
I have a Keras TextVectorization layer which uses a custom standardization function.
def custom_standardization(input_string, preserve=['[', ']'], add=['¿']):
strip_chars = string.punctuation
...
1
vote
0
answers
145
views
Load a TransformerBlock & TokenAndPositionEmbedding keras model after saving it
Good morning,
I hope you are well and will help me get out of this situation. Basically, I trained a tensorflow.keras model by having defined a TransformerBlock class as well as a ...
- 11
1
vote
0
answers
55
views
Low Tensorflow Model Test Accuracy
My model takes binary input which is from binary encoded text (not one-hot encoded). I achieve a binary accuracy of 99.5% and accuracy of 85%. Immediately after training, I achieve a test result of ...
- 11
0
votes
1
answer
110
views
How to format Ragged Tensor for Encoder-Decoder model?
I'm working on building a seq2seq model using encoder-decoder architecture for which I have built a tf.data.Dataset pipeline that reads the text from the directories, vectorizes using them tf.keras....
- 2,546
1
vote
0
answers
260
views
Node: 'sequential/embedding/embedding_lookup' indices[31,197] = 159947 is not in [0, 159943)
here's my neural network:
model = Sequential()
model.add(Input(shape=(max_len, )))
model.add(Embedding(input_dim = vocab_size, output_dim = embd_dim, input_length = max_len, weights = [...
- 27
0
votes
1
answer
106
views
Can't save trained transformer model
I have trained on Transformer model but cant save the best model. what is wrong here! Code is running fine and trained.
if not os.path.exists("asr-checkpoint"):
os.makedirs("asr-...
- 330
0
votes
0
answers
250
views
Keras ValueError: Paddings must be non-negative
I am trying to use the TokenAndPositionEmbedding layer inside my Keras model:
concatenate = keras.layers.concatenate(input_layers, axis=1)
# Batch size is 32, so the concatenate layer ouput shape is (...
1
vote
0
answers
192
views
Correctly submit triplets to a BERT model with triplet loss
I'm working on a model consisting in taking the elements of a triplet (consisting in an anchor, a positive example and a negative example), pass them through a BERT model and use them to calculate the ...
- 159
2
votes
1
answer
194
views
I have trained a custom transformer model on language modeling, now how do I make predictions with it?
I have trained a transformer model on language modeling (i.e predicting the next character given the context) on a dataset. CONTEXT_LENGTH = 200, I want the model to predict when the input is not of ...
- 79
5
votes
2
answers
689
views
How to improve the results of this neural network of finetuned BERT model?
I'm working on a NLP classification problem where I'm trying to classify training courses into 99 categories. I managed to make a few models including the Bayesian classifier but it had an accuracy of ...
- 11
0
votes
1
answer
104
views
Incorporating validation data correctly in model.fit with Keras and DistilBERT
I'm new to NLP and am trying to do some binary classification with DistilBERT on a Kaggle dataset (https://www.kaggle.com/competitions/nlp-getting-started/data). Everything is actually going fine, but ...
- 25
0
votes
1
answer
68
views
How to input a list into a Keras Model
I am pretty new to using tensowflow and keras.
I have an x_train and y_train set of the form
x_train = [[ 21 0 0 ... 0 0 0]
[ 22 0 0 ... 0 0 0]
[ 23 0 0 ... 0 0 0]
...
...
0
votes
0
answers
81
views
Why are my deep learning models giving unreasonably high accuracy on test data?
I'm trying to do sarcasm detection on Twitter data to replicate the results mentioned in this paper. Binary classification problem. For that I used a separate set of unlabeled tweets to create the ...
- 969
0
votes
1
answer
305
views
seq2seq inference outputs wrong results despite high accuracy
I am training a seq2seq model following Keras tutorial https://keras.io/examples/nlp/lstm_seq2seq/, the same code but a different dataset.
Here is the main model code for reference:
Code snippet for ...
- 1,201
0
votes
1
answer
464
views
How to save an embedding layer which was created outside model?
I created a word embedding layer outside model and used it as input before fitting my model. Now I need to predict new sentences by this model, how can I save the pre-trained embedding layer and apply ...
- 21
2
votes
2
answers
649
views
Constructing Tensorflow Dataset and applying TextVectorization layer using map method
I'm attempting to construct input to an embedding layer for an NLP model. However, I am having problems with converting raw text data to the numerical input required by the embedding layer.
Here is ...
- 737
3
votes
0
answers
2k
views
Unable to install tensorflow-text and unable to import keras_nlp
I am trying out the Keras-NLP library by using one of the examples provided on the Keras website. I have installed Keras-NLP using the command pip install keras-nlp and Tensorflow(version = 2.9.2).
...
- 431
0
votes
1
answer
196
views
BERT embeddings in LSTM model error in fit function
I am novice in TensorFlow
I am traying to use BERT embeddings in LSTM model
this is my model function
def bert_tweets_model():
Bertmodel = TFAutoModel.from_pretrained(model_name,...
- 39
0
votes
1
answer
87
views
Failed to convert a NumPy array to a Tensor (Unsupported object type list) error
I am doing a course on Natural Language processing using TensorFlow on coursera. In one of the labs, we have to explore overfitting in NLP. I am continuously getting the same error while running the ...
- 1
1
vote
1
answer
749
views
AttributeError: 'tuple' object has no attribute 'rank' when calling model.fit() in NLP task
I'm following this tutorial
https://towardsdatascience.com/another-twitter-sentiment-analysis-with-python-part-9-neural-networks-with-tfidf-vectors-using-d0b4af6be6d7
However, while implementing the ...
- 15
0
votes
0
answers
49
views
Sequential Keras model is able to predict different size of input?
I am new to recurrent networks and nlp. I try to create a text generation model. First I padded my sequences in order to prevent shape mismatch during training:
[1505, 422, 63, 324],
[ 7, 63, ...
- 1
1
vote
1
answer
502
views
Node: 'IteratorGetNext' - INVALID_ARGUMENT: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [5], [batch]: [0]
I'm trying to work on the Kaggle Getting Started Natural Language Processing with Disaster Tweets competition as an exam project for my uni deep learning course.
I am trying to solve the problem using ...
- 11
1
vote
0
answers
91
views
Is there a numerical "missing" token to tell LSTMs to impute a missing number?
I'm trying to impute missing spatial positions [x, y, z] from an object trajectory, using a bidirectional LSTM. To train, I've masked some known positions in trajectories with mask_value = -1, while ...
- 698
0
votes
1
answer
88
views
ValueError: Shapes (426530, 2) and (1930, 2) are incompatible for y_pred and y_test
I am working on the DistillBert project for binary classification. I am trying to run the following code using the Spam SMS data set (You can also use the IMDB dataset, it is also giving the same ...
0
votes
0
answers
39
views
Why am I getting the prediction value like [[ 8.45632 , -8.409305], [-8.977011, 8.996431],...] for a binary classification in Tersorflow?
I am working on the DistillBert project for binary classification. I am trying to run the following code using the Spam SMS data set (You can also use the IMDB dataset, it is also giving the same ...
0
votes
0
answers
169
views
Getting Invalid argument error for Binary classification
I am working on the DistillBert project for binary classification. I am trying to run the following code using the Spam SMS data set (You can also use the IMDB dataset, it is also giving the same ...
1
vote
1
answer
372
views
How to use Keras predict function for NLP models?
I have created an NLP classification model with keras with no problems with my model showing 83.5% accuracy upon evaluation. However, when I want to use my model to predict a new set of tokenized ...
- 41
0
votes
0
answers
110
views
Tensoflow: Input 0 of layer "conv1d_17" is incompatible with the layer: expected min_ndim=3, found ndim=2. Full shape received: (None, 16)
Although I could find similar questions, I have not been able to fix this issue.
I am trying to classify text, but got this error:
Input 0 of layer "conv1d_17" is incompatible with the layer:...
- 16
0
votes
1
answer
476
views
Tensorflow target shape not matching - how to properly format data
I'm trying to build NLP classifier, data consists of 2 columns, one with text other one represents target with 4 classes in total. I've one-hot encoded target, but when running the model.fit() method ...
- 13
1
vote
1
answer
221
views
Save best model with train_step without model.fit()
Hello I'm new in machine learning, so I'm trying to save the best model weights out of 30 epochs. Now I can only save all 30 models using this code
train_loss = tf.keras.metrics.Mean(name='train_loss')...
- 21
1
vote
1
answer
150
views
Why is model.fit working without clear attribute and label separation and the same method is not working for model.evaluate?
I am working on building DistillBERT model for IMDB dataset where the text is classified either as positive or negative. In my code I have first tokenised the 'text' data -
from datasets import ...
2
votes
1
answer
66
views
Custom text pre-processing saved in Tensorflow model
How to write custom text pre-processing that could be saved as part of a model?
Suppose that I would like to have two features:
auto-correct string input with some function. Words might change after ...
- 5,267
0
votes
1
answer
781
views
Overfitting on LSTM text classification using Keras
I am trying to develop an LSTM model using Keras, following this tutorial. However, I am implementing it with a different dataset of U.S. political news articles with the aim of classifying them based ...
- 3
1
vote
1
answer
436
views
Tensorflow 2.* - Get an internal Keras layer prediction values [duplicate]
I have a TensorFlow model like this-
I like to know the values of the red marked layer (5 float values) for the specific input to check how the model responds at this layer (attention layer). I need ...
- 14.5k
2
votes
2
answers
626
views
TimeDistributed(Dense()) vs Dense() after lstm
input_word = Input(shape=(max_len,))
model = Embedding(input_dim=num_words, output_dim=50, input_length=max_len)(input_word)
model = SpatialDropout1D(0.1)(model)
model = Bidirectional(LSTM(units=100, ...
- 307
0
votes
1
answer
267
views
Get top 3 prediction of LSTM instead of only the top
I have a LSTM model trained on text content. And now I want to use that model to generate some sentences. But instead of always picking the best option, i want it to select from for example the top 3, ...
- 105
0
votes
1
answer
120
views
Write generator function for LSTM text generation model
i have a LSTM model for text generation but when trying to increase the amount of data to input, I run into RAM issues so I found out that I can use fit_generator function to load the data in step by ...
- 105
2
votes
1
answer
6k
views
Tensorflow: InvalidArgumentError: Graph execution error:
I have the following preprocessing for a tensorflow neural-network:
import csv
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import ...
- 8,371
0
votes
1
answer
354
views
Word2Vec + CNN Overfitting
Currently I'am training my Word2Vec + CNN for Twitter sentiment analysis about COVID-19 vaccine domain. I used the pre-trained GoogleNewsVectorNegative300 word embedding. The problem is why I heavily ...
0
votes
1
answer
784
views
Word2Vec + LSTM Good Training and Validation but Poor on Test
currently I'am training my Word2Vec + LSTM for Twitter sentiment analysis. I use the pre-trained GoogleNewsVectorNegative300 word embedding. The reason I used the pre-trained ...