-
Notifications
You must be signed in to change notification settings - Fork 0
Retrieving Knowledge From Code with Code Word Clouds
Alexandru Bolboaca edited this page Jun 24, 2020
·
3 revisions
One (simple) way to harvest some knowledge from code is to look at the words used in the code. These are the names of classes, methods and fields.
Here is a static example of usual words in large C# code base/libraries: Words used in Source Code.
A tool I have been using is Source Code Word Cloud Generator @ CodePlex. The overview page (previous link has good explainations).
So if you take your code remove comments, literals, block some very common words (like company name) and generate a word cloud of it, you will get an interesting picture to discuss with your colleagues in a coffee corner.
- If words "if", "then", "else", "switch", "case" are first what you see - your code is sprinkled with conditionals!
- Is "string" in your words top 10 ? - Congratulations if you write text processing software, otherwise in might be a bad smell.
- Are you writing API or a library so you should see word "public" in front rows. If you are not working on a library or API, the word public might be a signal to think on better protection.
- Do you see your classes at first glance or are they far away in background? Behind "int", "byte", "array" etc.? Is your code in your domain language?
(Alex) I played a little with command line tools to get a list of words. Needs a bit more work, but this works on a C++ code sample:
#!/usr/bin/env bash
FOLDER=PATH_TO_SOURCE_CODE_FILES
FILE=./temp.txt
KEYWORDS=../word_cloud/keywords.txt
# Remove comments from C++ code
g++ -fpreprocessed -dD -E -P -w $FOLDER/* > $FILE
# Get all words and sort them
grep -o -E '\w+' $FILE | sort -u -f -o all_words.txt
# Eliminate keywords
grep -v -f $KEYWORDS all_words.txt > words.txt
Still need to:
- eliminate #includes
- eliminate more words from STL
- preserve full string constants
The next step would then be to apply some natural language processing to identify synonyms.
Can you guess what this set of source files does? :)
0
0f
1
100
114
120
1280
140
180
2
255
720
82
93
a
Action
addComponent
add_option
allVdfsPackages
AlphaMasked
also
ambientOcclusion
Application
argc
argument
argv
asset
ASSETS
assign
at
BC_1
BC_A
BC_D
BC_DOWN
BC_E
BC_F5
BC_LCONTROL
BC_LEFT
BC_LMENU
BC_LSHIFT
BC_MOUSE_LEFT
BC_Q
BC_RIGHT
BC_RSHIFT
BC_S
BC_UP
BC_W
BC_X
be
been
bs
BsApplication
BsCCamera
BsFileSystem
BsPath
BsPrefab
BsPrerequisites
BsSceneObject
BsVirtualInput
BsZenLib
but
Cache
cached
Caching
called
Camera
Can
cassert
CCamera
Character
CharacterKeyboardInput
components
config
content
contentPath
could
create
createGlobal
c_str
cullDistance
current
cxxopts
determined
Did
either
EmptyGame
enableAutoExposure
enabled
enableFXAA
enableHDR
enableIndirectLighting
enableSkybox
enableTonemapping
Engine
EngineConfig
EngineContent
engineExecutablePath
exception
executable
EXIT_FAILURE
EXIT_SUCCESS
Expected
exposureScale
f
Failed
FastMove
Fatal
files
FileSystem
final
find
findChild
findEngineContent
findGameFilesRoot
Finding
first
folder
follow
Found
fullscreen
g
GAME
GameplayUI
gameType
gamma
get
getComponent
getConfiguration
getFilename
getRenderSettings
given
Giving
Gothic
Gothic1
Gothic1Game
Gothic2
Gothic2Game
gVirtualFileSystem
gVirtualInput
h
has
hasFoundGameFiles
HCamera
HCharacter
hero
heroSO
HPrefab
HSceneObject
HThirdPersonCamera
II
Image
in
Indexing
Info
initializeBsf
initialized
Input
InputAxis
inputConfig
insertCharacter
installation
instance
instantiate
InvalidParametersException
InvalidStateException
iostream
is
isEmpty
isFullscreen
isStarted
it
Jump
load
loaded
loadGamePackages
Loading
loadModPackages
loadPackage
loadShaders
log
logging
Main
mainloop
makeAbsolute
mConfig
mEngineContent
mMainCamera
mode
MouseX
MouseY
move
MoveBack
mThirdPersonCamera
NEWWORLD
No
Note
now
only
Opaque
Options
OriginalGameFiles
OriginalGameResources
override
p
packages
parseArguments
parse_positional
Path
PC_HERO
plane
populateCache
populateResourceCache
positional
positional_help
Possible
PX
QuickSave
Received
REGoth
REGOTH_CONTENT_DIR_NAME
REGOTH_LOG
REGOTH_THROW
render
RenderMode
request
res
resolution
resolutionX
resolutionY
resource
resources
Rotate
rs
run
RunEngine
runInitScripts
runMainLoop
Running
runs
save
SAVEGAME
Saving
Scene
SceneCamera
sceneCameraSO
SceneObject
screenSpaceReflections
set
setActive
setMain
setMSAACount
setPathToEngineExecutable
setRenderSettings
setShaders
Setting
setupInput
setupMainCamera
setupScene
ShaderKind
Shaders
show_positional_help
Shutting
Sky
skyRenderMode
SO
SPtr
startUp
str
StrafeLeft
StrafeRight
String
StringUtil
Succeded
supplied
T
the
ThirdPersonCamera
Throw
to
ToggleMeleeWeapon
ToggleSneaking
ToggleWalking
toLowerCase
toString
tpp
Transparent
Try
Trying
TurnLeft
TurnRight
UINT32
unique_ptr
up
useAsHero
used
userInput
value
values
VDFS
verbosity
Vertical
Video
VideoMode
vidgrp
VIRTUAL_AXIS_DESC
VirtualFileSystem
Warning
with
WORLD
WORLD_STARTPOINT
x
y
yet
ZEN