Gfyuki

I learned how to build recurrent cnn model for text classification and sketched out my initial implementation. However, I am wondering how to transform recurrent cnn model for sentence classification. I am curious how can I come up better implementation of recurrent cnn model for sentence classification task. Here is part of keras solution that I used:

import gensim

import numpy as np

import string

import gensim

from gensim.models import Word2Vec

from gensim.utils import simple_preprocess

from gensim.models.keyedvectors import KeyedVectors



word2vec = KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', limit= 500000,binary=True)

embeddings = np.zeros((word2vec.syn0.shape[0] + 1, word2vec.syn0.shape[1]), dtype = "float32")

embeddings[:word2vec.syn0.shape[0]] = word2vec.syn0



MAX_TOKENS = word2vec.syn0.shape[0]

embedding_dim = word2vec.syn0.shape[1]

hidden_dim_1 = 200

hidden_dim_2 = 100

NUM_CLASSES = 10

problem

I want to learn sentence classification task by using recurrent cnn (RCNN) model. The fact that some people used RCNN for object recognition problem. And it is not very intuitive for me how to transform same idea to list of short sentences.

Here is the code that I want to make them work for sentence classification task:

document = Input(shape = (None, ), dtype = "int32")

left_context = Input(shape = (None, ), dtype = "int32")

right_context = Input(shape = (None, ), dtype = "int32")



embedder = Embedding(MAX_TOKENS + 1, embedding_dim, weights = [embeddings], trainable = False)

doc_embedding = embedder(document)

l_embedding = embedder(left_context)

r_embedding = embedder(right_context)

continuation of my code

I am struggling to make above code in problem section for sentence classification problem. Can anyone give me possible idea how to make it work for sentences classification?

If there is efficient transformation on above code, I'd like to continue my pipeline as follow to build RCNN model for sentence classification.

forward = LSTM(hidden_dim_1, return_sequences = True)(l_embedding)

backward = LSTM(hidden_dim_1, return_sequences = True, go_backwards = True)(r_embedding)

backward = Lambda(lambda x: K.reverse(x, axes = 1))(backward)

together = concatenate([forward, doc_embedding, backward], axis = 2)

semantic = Conv1D(hidden_dim_2, kernel_size = 1, activation = "tanh")(together)

pool_rnn = Lambda(lambda x: K.max(x, axis = 1), output_shape = (hidden_dim_2, ))(semantic)

model_output = Dense(NUM_CLASSES, input_dim = hidden_dim_2, activation = "softmax")(pool_rnn)

model_RCNN = Model(inputs = [document, left_context, right_context], outputs = model_output)

maybe I need to tokenize all sentences and create array for right/left context for each sentences, but I didn't get solid idea on that. Any more thoughts?

question

how can I realistically create input matrix for sentences list, right/left context of each sentence? Any workaround to get this done? Any efficient sketch solution to use recurrent cnn model for sentence classification? Thanks in advance!