Some Python NLP Modules

Here I collect some python modules for NLP

text vector, text similarity

• Chinese text to vector, text vectorization tools, including word vectorization and sentence vectorization
• https://www.ctolib.com/shibing624-simtext.html
• https://github.com/shibing624/text2vec

Example

from text2vec import SearchSimilarity

a = '如何更换花呗绑定银行卡'
b = '花呗更改绑定银行卡'
c = '我什么时候开通了花呗'

corpus = [a, b, c]
print(corpus)
search_sim = SearchSimilarity(corpus=corpus)

print(a, 'scores:', search_sim.get_scores(query=a))
print(a, 'rank similarities:', search_sim.get_similarities(query=a))

word extract

• https://github.com/fighting41love/funNLP
• https://github.com/fighting41love/cocoNLP

word segmentation toolkit

• https://github.com/lancopku/pkuseg-python
• http://thulac.thunlp.org/

Example

代码示例1
import thulac   

thu1 = thulac.thulac()  #默认模式
text = thu1.cut("我爱北京天安门", text=True)  #进行一句话分词
print(text)

How To Install Docker In Windows10

Processing

As my windows10 version is 1607 & os version is 14393, I cannot install Docker for Windows Installer, but you can use Docker Toolbox as substitution.

If you try to install Docker for Windows Installer, you will got an error as bellow.

You can also try to installing at Windows10 Home, here is the blog.

In order to use docker & docker-compose in my windows10, I installed Docker Toolbox, and succeeded finally.

Otherwise, you had to enable VT-X/AMD-v in BIOS, or you will got this error.

In addition, you should diable Hyper-V

After that, you can try again to run start.sh.

Reference

• https://www.jianshu.com/p/68fd8a072622
• https://www.cnblogs.com/shaosks/p/6932319.html
• https://www.ibm.com/mysupport/s/question/0D50z000055jiLRCAY/creating-the-virtual-machine-virtualbox
• https://itnext.io/install-docker-on-windows-10-home-d8e621997c1d

Git Manual

Git global setup

git config --global user.name "username"
git config --global user.email "your@email.com"

Create a new repository

git clone git@gitlab.com:username/Repository.git
cd Repository
touch README.md
git add README.md
git commit -m "add README"
git push -u origin master

Existing folder

cd existing_folder
git init
git remote add origin git@gitlab.com:username/Repository.git
git add .
git commit -m "Initial commit"
git push -u origin master

Existing Git repository

cd existing_repo
git remote rename origin old-origin
git remote add origin git@gitlab.com:username/Repository.git
git push -u origin --all
git push -u origin --tags

Update Codes To Gitlab

Solution：

• open terminal, locate to the projection.
• input：git status，if red line exist mean file changed.
• input：git commit -a -m "tag"，add commits to files.
• input：git push， update codes to git server.

Masscan Basic Manual

scan ip range, port list

need to specify parameters adapter-ip、adapter-mac、router-mac to reduce network interference

sudo masscan 192.168.0.0/16 -p445,22,3306,6379,2375,6380,9001,873,7001 --adapter-ip 172.16.6.132 --adapter-mac 1c-36-bb-ed-0e-13 --router-mac 78-44-fd-0c-87-fb -Pn --max-rate=500 -oX ~/Desktop/masscan_scan.xml

port scan for a single ip

sudo masscan 112.35.18.166 -p20-65534 --adapter-ip 172.16.6.132 --adapter-mac 1c-36-bb-ed-0e-13 --router-mac 78-44-fd-0c-87-fb -Pn --max-rate=300 -v

Tmux Basic Manual

Usage

1. create session

tmux

2. create and name session

tmux new -s $session_name

3. exit session tempararyly

Ctrl+b d

4. list session

tmux ls

5. enter existing session

tmux a -t $session_name

6. create window

Ctrl+b c

7. delete window

Ctrl+b &

8. get list of existing sessions

Ctrl-b s

reference

• https://blog.csdn.net/qq_37468954/article/details/82178036