不同尺寸的模型中比较Word2Vec向量
创始人
2025-01-08 10:00:23
0

要比较不同尺寸的Word2Vec向量模型,可以按照以下步骤进行操作:

  1. 导入所需的库:
from gensim.models import Word2Vec
import numpy as np
  1. 定义训练数据和模型参数:
sentences = [['this', 'is', 'the', 'first', 'sentence'],
            ['this', 'is', 'the', 'second', 'sentence'],
            ['yet', 'another', 'sentence'],
            ['one', 'more', 'sentence'],
            ['and', 'the', 'final', 'sentence']]

model_size1 = Word2Vec(sentences, size=100, window=5, min_count=1)
model_size2 = Word2Vec(sentences, size=200, window=5, min_count=1)
  1. 获取词汇表和对应的向量:
vocab_size1 = len(model_size1.wv.vocab)
vocab_size2 = len(model_size2.wv.vocab)

vocab1 = list(model_size1.wv.vocab.keys())
vocab2 = list(model_size2.wv.vocab.keys())

vectors1 = np.stack([model_size1.wv[word] for word in vocab1])
vectors2 = np.stack([model_size2.wv[word] for word in vocab2])
  1. 比较向量:
# 计算两个模型的平均向量
mean_vector1 = np.mean(vectors1, axis=0)
mean_vector2 = np.mean(vectors2, axis=0)

# 计算两个模型的相似度
similarity = np.dot(mean_vector1, mean_vector2) / (np.linalg.norm(mean_vector1) * np.linalg.norm(mean_vector2))

完整代码示例:

from gensim.models import Word2Vec
import numpy as np

sentences = [['this', 'is', 'the', 'first', 'sentence'],
            ['this', 'is', 'the', 'second', 'sentence'],
            ['yet', 'another', 'sentence'],
            ['one', 'more', 'sentence'],
            ['and', 'the', 'final', 'sentence']]

model_size1 = Word2Vec(sentences, size=100, window=5, min_count=1)
model_size2 = Word2Vec(sentences, size=200, window=5, min_count=1)

vocab_size1 = len(model_size1.wv.vocab)
vocab_size2 = len(model_size2.wv.vocab)

vocab1 = list(model_size1.wv.vocab.keys())
vocab2 = list(model_size2.wv.vocab.keys())

vectors1 = np.stack([model_size1.wv[word] for word in vocab1])
vectors2 = np.stack([model_size2.wv[word] for word in vocab2])

mean_vector1 = np.mean(vectors1, axis=0)
mean_vector2 = np.mean(vectors2, axis=0)

similarity = np.dot(mean_vector1, mean_vector2) / (np.linalg.norm(mean_vector1) * np.linalg.norm(mean_vector2))

print("Model 1 vocab size:", vocab_size1)
print("Model 2 vocab size:", vocab_size2)
print("Similarity between the two models:", similarity)

这样就可以比较不同尺寸的Word2Vec向量模型,并计算它们的相似度。

相关内容

热门资讯

透视教程!wpk透视表,hhp... 透视教程!wpk透视表,hhpoker透视脚本,一分钟教会你(有挂介绍)1、打开德州poker外挂分...
透视规律!wpk透视表,hhp... 透视规律!wpk透视表,hhpoker德州挂真的有吗,玩家教程(一贯存在有挂);玩家必备必赢加哟《1...
透视工具!德普之星透视辅助软件... 【福星临门,好运相随】;透视工具!德普之星透视辅助软件,哈糖大菠萝怎么挂,2025版教程(原来真的有...
透视玄学!wepoekr底牌透... 透视玄学!wepoekr底牌透视,德普之星透视辅助软件,必备科技(有挂攻略)1、玩家可以在德普之星透...
透视有挂!wpk辅助插件,hh... 《透视有挂!wpk辅助插件,hh poker辅助有用吗,必赢方法(从来有挂)》 wpk辅助插件软件透...
透视ai!德普之星透视辅助,w... 您好,德普之星透视辅助这款游戏可以开挂的,确实是有挂的,需要了解加Q群【1067239143】很多玩...
透视私人局!wepoker透视... 透视私人局!wepoker透视脚本,werplan怎么透视,揭秘教程(切实真的是有挂)1、完成wep...
透视智能ai!aapoker透... 透视智能ai!aapoker透视插件,智星菠萝可以辅助吗,推荐一款(有挂教程)1、在智星菠萝可以辅助...
透视智能ai!wpk脚本下载,... 1、透视智能ai!wpk脚本下载,智星德州插件2024最新版,2025新版教程(其实是真的有挂);详...
透视讲解!wepoker透视脚... 透视讲解!wepoker透视脚本下载,红龙poker有辅助吗,一分钟了解(有挂技巧)1、玩家可以在w...